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ACQUISITION OF MAJOR WEAPONS SYSTEMS 
BY THE DEPARTMENT OF DEFENSE AND 
S. 454, THE WEAPON SYSTEMS ACQUISITION 
REFORM ACT OF 2009 


TUESDAY, MARCH 3, 2009 

U.S. Senate, 

Committee on Armed Services, 

Washington, DC. 

The committee met, pursuant to notice, at 9:36 a.m. in room SH- 
216, Hart Senate Office Building, Senator Carl Levin (chairman) 
presiding. 

Committee members present: Senators Levin, Lieberman, Akaka, 
Bill Nelson, Webb, McCaskill, Udall, Hagan, Begich, Burris, 
McCain, Chambliss, Thune, Martinez, Burr, and Collins. 

Committee staff members present: Richard D. DeBobes, Staff Di- 
rector, and Leah C. Brewer, Nominations and Hearings Clerk. 

Majority staff members present: Creighton Greene, professional 
staff member; Peter K. Levine, general counsel; John H. Quirk V, 
professional staff member; Arun A. Seraphin, professional staff 
member; and William K. Sutey, professional staff member. 

Minority staff members present: Joseph W. Bowab, Republican 
staff director; Daniel A. Lerner, professional staff member; David 
M. Morriss, minority counsel; Lucian L. Niemeyer, professional 
staff member; and Christopher J. Paul, professional staff member. 

Staff assistants present: Jessica L. Kingston, Brian F. Sebold, 
and Breon N. Wells. 

Committee members’ assistants present: Jay Maroney, assistant 
to Senator Kennedy; Bonni Berge, assistant to Senator Akaka; 
Christopher Caple, assistant to Senator Bill Nelson; Jon Davey, as- 
sistant to Senator Bayh; Gordon 1. Peterson, assistant to Senator 
Webb; Stephen C. Hedger, assistant to Senator McCaskill; Jennifer 
Barrett, assistant to Senator Udall; Michael Harney, assistant to 
Senator Hagan; David Ramseur, assistant to Senator Begich; 
Brady King, assistant to Senator Burris; Sandra Luff, assistant to 
Senator Sessions; Clyde A. Taylor IV, assistant to Senator 
Chambliss; Jason Van Beek, assistant to Senator Thune; Erskine 
W. Wells III, assistant to Senator Wicker; and Kevin Kane, assist- 
ant to Senator Burr. 

OPENING STATEMENT OF SENATOR CARL LEVIN, CHAIRMAN 

Chairman Levin. Good morning, everybody. The committee 
meets today to consider the performance of the Department of De- 
fense’s (DOD) acquisition programs at a time when cost growth on 
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these programs has reached levels that we cannot afford, including 
consideration of our bill, S. 454, the Weapon Systems Acquisition 
Reform Act of 2009, which Senator McCain and I recently intro- 
duced. 

Since the beginning of 2006, nearly half of the DOD’s largest ac- 
quisition programs have exceeded the so-called Nunn-McCurdy 
cost-growth standards established by Congress to identify seriously 
troubled programs. 

As Secretary Gates pointed out in his testimony before our com- 
mittee last month, the list of big-ticket weapon systems that have 
experienced contract or program performance problems spans the 
Services and includes the Air Force tanker, the CSAR-X, the VH- 
71, the Osprey, the Future Combat Systems (FCS), the Armed Re- 
connaissance Helicopter, the Littoral Combat Ship (LCS), the Joint 
Strike Fighter (JSF), and so on. 

Overall, DOD’s 95 defense acquisition programs, known as Major 
Defense Acquisition Programs (MDAPs), have exceeded their re- 
search and develop budgets by an average of 40 percent, seen their 
acquisition costs grow by an average of 26 percent, and experienced 
an average schedule delay of almost 2 years. Last summer, the 
Government Accountability Office (GAO) reported that cost over- 
runs on DOD’s MDAPs now total $295 billion over the original pro- 
gram estimates. That’s true, even though we’ve cut unit quantities 
and reduced performance expectations on many programs, in an ef- 
fort to hold costs down. 

These cost overruns happen because of fundamental flaws that 
are endemic to our acquisition system. We have a pretty good idea 
of what those flaws are. Major acquisition programs fail because 
DOD: one, continues to rely on unreasonable cost and schedule es- 
timates; two, establishes unrealistic performance expectations; 
three, insists on the use of immature technologies; and four, adopts 
costly changes to program requirements, production quantities, and 
funding levels in the middle of ongoing programs. 

Earlier this year. Under Secretary of Defense for Acquisition 
John T. Young, Jr., wrote a memo in which he sought to explain 
the cost growth on some of DOD’s largest programs. This is what 
his memorandum said: “A number of programs had a poor founda- 
tion and Milestone B, the starting point for major development and 
manufacturing design. . . Fundamentally, these programs moved 
past Milestone B with inadequate foundations built on artificially 
low cost estimates, optimistic schedules and assumptions, imma- 
ture design or technology, fluid requirements, and other issues.” 

Mr. Young then went on to list the flaws of each MDAP. The 
JSF: too little understanding of the design; the FCS: fluid program 
strategy; the V-22: immature technology and Congress reversed 
DOD termination; the C-17: development issues and underfunding; 
the Army’s Family of Medium Tactical Vehicles: design flaws; the 
CH-47F: low estimates and invalid remanufacture assumptions; 
the advanced EHF Satellite: optimistic schedule; the LPD-17: 
flawed lead ship design process and knowledge base; and the F- 
22A: immature, exquisite technology. 

Now, the first two of these programs, JSF and FCS, account for 
almost $80 billion in cost overruns, with average unit costs that 
have already increased by roughly 40 percent each over original 



3 


program estimates, and are likely to rise further. According to 
GAO, both programs were initiated with insufficiently mature tech- 
nologies and overly optimistic assumptions about system perform- 
ance. 

With regard to the JSF, GAO reports that initial estimates as- 
sumed that commonality between the three variants of the aircraft 
could cut development costs by about 40 percent; however, this 
level of commonality has proven impossible to achieve. Twelve 
years after the program started, three of the JSF’s eight critical 
technologies are still not mature, its production processes are not 
mature, and its designs are still not fully proven and tested. 

With regard to FCS, GAO reports that the estimated lines of 
code needed to support FCS’s software and development are almost 
three times the original assumptions, leading to an increase in soft- 
ware development costs that now approaches $8 billion. Eight 
years after the program started, only 3 of the FCS’s 44 critical 
technologies are fully mature. GAO tells us that the Army has not 
advanced the maturity of 11 critical technologies since 2003, and 
that 2 other technologies, which are central to the Army’s plans, 
are now rated less mature than when the program began. 

This is the price that we have paid for our failure to complete 
needed systems engineering tasks, perform appropriate develop- 
mental testing, and build prototypes. Particularly at this time, 
when the Federal budget is under immense strain as a result of the 
economic crisis, we cannot continue this kind of waste and ineffi- 
ciency. 

That is why Senator McCain and I have introduced the Weapon 
Systems Acquisition Reform Act of 2009. This bill is designed to 
help put MDAPs on a sound footing from the outset by addressing 
program shortcomings in the early phases of the acquisition proc- 
ess. 

In particular, our bill would address unreasonable performance 
requirements by requiring DOD to rebuild its systems engineering 
capability, reestablish the position of director of developmental 
testing, and use the Joint Requirements Oversight Council (JROC) 
to make early tradeoffs between cost schedule and performance re- 
quirements. 

Our bill will address unreasonable cost and schedule estimates 
by establishing a new director of independent cost assessment to 
ensure that cost estimates for MDAPs are fair, reliable, and unbi- 
ased. 

Our bill will reduce the use of immature technologies by requir- 
ing DOD to make greater use of prototypes, including competitive 
prototypes, and requiring the Director of Defense Research and En- 
gineering to periodically review and provide independent assess- 
ments of the maturity of critical technologies on major weapon sys- 
tems. 

Our bill, finally, addresses costly changes in the middle of pro- 
grams by ensuring, through preliminary design review, that re- 
quirements are well understood before a program receives Mile- 
stone B approval, by providing an incentive for contractors to im- 
prove performance in ongoing programs by developing mechanisms 
to maintain competitive pressure through the program cycle, and 
by tightening the so-called Nunn-McCurdy requirements for under- 
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performing programs by providing for the termination of any pro- 
gram that cannot be justified after a complete reexamination and 
revalidation. 

Today, we will hear from two distinguished former Under Secre- 
taries of Defense for Acquisition, Paul Kaminski and Jacques 
Gansler. We will also hear from Pete Adolph, a former DOD Direc- 
tor of Developmental Testing, and Mike Sullivan, the GAO Director 
of Acquisition and Sourcing Management. Each of our witnesses 
has great experience in the area of weapon systems acquisition; 
and, in the course of the last year, each has completed a major re- 
port recommending significant improvements and reforms. We all 
look forward to their testimony on these issues. 

I now call on Senator McCain. 

STATEMENT OF SENATOR JOHN McCAIN 

Senator McCain. Thank you, Mr. Chairman, for today’s hearing, 
and more importantly, your leadership on the bill that is the sub- 
ject of today’s hearing, the Weapons Acquisition Reform Act of 
2009. I join you in welcoming our expert witnesses today. 

Let me set the overall context of today’s discussion, and I’ll do 
so very simply. A train wreck is coming. Look at the President’s 10- 
year budget and you’ll see an overall decrease in defense spending. 
Unless difficult decisions are made and serious reform measures 
are undertaken, our ability to provide for our national security will 
be, over time, fundamentally compromised. 

Clearly, the endless cycle of runaway costs, prolonged delivery 
schedules, and poor performance in the acquisition of major weap- 
ons has, in my view, us mired in a form of unilateral disarmament. 
Since scrutinizing the tanker lease scandal years ago. I’m not sure 
that things are any better. For example, how could DOD award a 
multibillion-dollar contract based on a proposal it later found was 
fundamentally unexecutable? That’s exactly what happened on the 
Navy’s VH-7i program, the program to replace the President’s own 
helicopters. 

Just over the last few years, the VH-71 program has doubled, 
with an additional cost of $6 billion for 28 aircraft that will likely 
cost taxpayers well over $400 million each. How could DOD laden 
a multibillion-dollar shipbuilding program with so many require- 
ments that the program more than doubled in cost, with DOD basi- 
cally asleep at the switch? That happened with the Navy’s LCS 
program. At times, the program saw change orders averaging 75 
per week. 

How could a multibillion-dollar program for next-generation 
fighter jets produce planes that are operating below satisfactory 
readiness rates and could end up being too expensive to operate? 
That happened with the Air Force’s F-22 Raptor program. 

How could DOD spend billions for the Army’s biggest trans- 
formational program, valued at almost $200 billion, only for it to 
be, in many respects, closer to the beginning of development than 
it is to the end? That’s the FCS program. At this point, it’s not 
been clear when, or even if, the information network at the heart 
of the FCS concept can be built. 

On our military satellite program, how could a design flaw re- 
cently emerge that will take at least 1 year, and up to $1 billion 
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to fix? That’s the Air Force’s Space-Based Infrared System (SBIRS), 
high satellite program. More cost and schedule increases are likely 
there. 

But, to understand the depth and breadth of our acquisition 
problems, one needs to go no further than to look at the status of 
particular programs. Across all the Services, the top 75 programs 
have unfunded cost overruns of at least $295 billion. From 2000, 
the number of MDAPs has increased from 75 to 95, and the cost 
of those programs has doubled from $790 billion to $1.6 trillion, 
leaving unfunded acquisition commitments equal to more than 10 
years’ worth of major weapons procurement funding. 

In other words, in the current fiscal environment we find our- 
selves, the DOD acquisition plan is unaffordable. In my view, 
meaningful reform is only going to happen if DOD itself decides to 
change, develops an overarching management philosophy, sets up 
clear lines of authority and accountability, brings discipline and 
control over the requirement process, shuts the revolving door, and 
restores the corps of qualified and experienced acquisition and con- 
tracting professionals. That’s what this legislation helps to do. 

In this bill, the chairman and I built on previous reform initia- 
tives by focusing on costs and risk. The bill reflects that a key to 
managing defense procurement programs effectively is starting 
them right by requiring key program reviews up front to catch cost- 
ly design flaws and technology risks before we actually buy them. 

Probably the most aggressive feature of the bill gives DOD a big 
stick, bigger than anything available under current law, to wield 
against the very worst-performing programs. It does so by giving 
DOD additional tools to enforce fair, reliable, and unbiased inde- 
pendent cost estimates with the creation of a new director. Unlike 
merely promulgated DOD instructions, which apply only to new 
programs, that provision will capture chronically-poor performers 
that are in the development pipeline now. 

Mr. Chairman, I don’t want to go all over the features of the bill; 
I want to hear from our witnesses. But, for truly meaningful re- 
form to endure, the commitment to reform must begin with the fi- 
duciaries of the taxpayers dollars within the DOD itself. 

I thank you, Mr. Chairman. I welcome the witnesses. 

[The prepared statement of Senator McCain follows:] 

Pkepared Statement by Senator John McCain 

Thank you, Mr. Chairman, for today’s hearing and, more importantly, your lead- 
ership on the hill that is the subject of today’s hearing, S.454, “The Weapons Sys- 
tems Acquisition Reform Act of 2009.” 

I appreciate the opportunity to serve as an original co-sponsor on it with you, and 
I too welcome our expert witnesses today. 

Before I turn to the bill, let me briefly set the overall context of today’s discussion, 
and I’ll do so simply. 

A train wreck is coming, and unless hard decisions are made and serious reform 
measures undertaken, our ability to see to our national security interests will be 
over time fundamentally compromised. 

Let me be clear. The endless cycle of runaway costs, prolonged delivery schedules 
and poor performance we have seen in major weapons has us, in my view, mired 
in a form of unilateral disarmament. 

Since we closely scrutinized the tanker lease scandal years ago, I’m not sure that 
things have gotten much better. For example: 

• How could the DOD have awarded a multibillion contract based on a pro- 
posal it later found was “fundamentally unexecutable.” That’s exactly what 
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happened on the Navy’s VH-71 program — the program to replace the Presi- 
dent’s own helicopters. Just over the last few years, the program has in- 
creased by about $6 billion — for aircraft that will likely cost teixpayers well 
over $400 million each. 

How could the DOD have loaded up a multibillion shipbuilding program with so 
many requirements that the program doubled in cost (by about $400 million) with- 
out the DOD really noticing until it was too late? That happened — on the Navy’s 
Littoral Combat Ship (LCS) program. At times, that program saw change orders 
averaging 76 per week. 

• How could a multibillion dollar program for next generation fighter jets 
have produced planes that may end up being too expensive to operate? That 
too happened — on the F-22 Raptor program. While being the Nation’s most 
expensive fighter aircraft, those jets continue to operate below satisfactory 
reliability rates. 

• How could the DOD have spent billions for the Army’s biggest trans- 
formational program, valued at about $200 billion, only for it to be (in many 
respects) closer to the beginning of development than it is to the end? 
That’s the Future Combat Systems (FCS) program. At this point, it’s not 
even clear when (or even iD the information network at the heart of the 
FCS concept can be built. 

• On a military satellite program, how could a design flaw have recently 
emerged that will take at least 1 year and up to $1 billion to fix? That’s 
the Air Force’s Space-Based Infrared System High satellite program. More 
cost and schedule increases are likely there. 

We’re supposed to have laws in the books that are supposed to prevent these sorts 
of things from happening. Why didn’t they work? To the person — who is responsible? 

The fact that we’re asking those questions (with billions of dollars of taxpayer 
money at stake) and that we have no answers, lays out what we’re dealing with far 
better than I can possibly describe using facts and figures. 

So, I don’t have to mention that the top 75 programs across all of the Services 
have unfunded cost overruns of at least $295 billion. Or, that (since fiscal year 2000) 
the number of major defense acquisition programs has increased from 75 to 95. Or, 
that within that period the cost of those programs doubled from $790 billion to $1.6 
trillion. Or, that this left unfunded acquisition commitments equal to more than 10 
years worth of major weapons procurement funding. 

I don’t have to talk about how risky developing most of those programs are; or 
the likelihood that they too will balloon in costs; or how much other government- 
wide priorities will constrain defense spending going forward. I don’t have to go into 
all that to make the point that the DOD’s acquisition plan — as it currently stands — 
is itself likely unaffordable. 

However one looks at it, the honeymoon is over. 

In my view, meaningful reform is only going to happen if DOD itself decides to 
change. DOD has to: 

• Develop an overarching management philosophy that dictates an overall 
approach to ensuring the timely delivery of major weapons that satisfy the 
needs of the joint warfighter at the most reasonable cost to the teixpayer. 

• Set up clear lines of authority and accountability for managing procure- 
ment programs. 

• Bring discipline and control over the requirements process and get out of 
the business of gold-plating programs. 

• Shut the revolving door. While a leavening of experienced DOD procure- 
ment officials working for defense contractors (and vice-versa) is healthy, 
the lack of meanin^ul controls on this revolving door is creating an 
unhealthy tolerance of conflict-of-interest. 

• Restore the corps of qualified and experienced acquisition and contracting 
professionals that DOD had in the 1980s before it gave its functions over 
to contractor/lead systems integrators, thereby letting the fox guard the hen 
house. 

Until and unless administration and DOD leadership do these sorts of things — 
things that set, if you will, a “command climate” that’s conducive to investing the 
taxpayers’ money responsibly — I fear that reform efforts may amount to only rear- 
ranging the deck chairs on the Titanic. That’s something that the chairman and I 
have no interest in doing. 

At the end of the day, we in Congress can only give DOD tools that it can use 
to pursue truly lasting solutions that ensure desirable cost/scheduling and perform- 
ance outcomes in our most complicated, most expensive weapons systems. That’s 
what this bill helps do. 
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In this bill, the chairman and I build on previous reform initiatives by focusing 
on costs and risk. The bill reflects that a key to managing defense procurement pro- 
grams effectively is starting them right — by requiring key program review upfront 
to catch costly design flaws and technology risk before we actually buy the system. 

In so doing, we continue our efforts to move DOD closer towards fixed price-type 
procurement contracting by requiring that technology and integration risk can be 
meted out of a program early. So, by the time a program heads into procurement, 
if a contractor isn’t ready to sign a good fixed price-type contract, the government 
shouldn’t be signing a contract with that company to buy that system at all. There’s 
still too much risk. 

Probably the most aggressive feature of the bill gives DOD a big stick — bigger 
than anything available under current law — to wield against the very worst per- 
forming programs. With that provision we intend DOD to, if you will, “enforce” fair, 
reliable, and unbiased cost estimates verified by a new director of independent cost 
estimates, also created by this bill. Unlike rules that DOD recently put in place, 
that provision doesn’t apply to just new programs. It will capture chronically poor 
performers that are in the development pipeline now. 

Another important provision requires DOD to consider a broad range of cost-effec- 
tive measures to help maximize competition throughout the life of a weapons pro- 
gram. The industry consolidation that occurred after the end of the Cold War went 
too far. Some 50 prime contractors merged into only 6. That’s far too few to support 
a competitive base for our current and future needs. It’s resulted in a serious decline 
in innovation. 

Other provisions in the bill elaborate in the “starting programs right” theme by: 

• Renewing focus on systems engineering early; 

• Requiring the completion of preliminary design reviews before a program 
can move into the development phase; and 

• Strengthening DOD’s developmental testing and evaluation capability. 

Other helpful provisions include those that: 

• Require DOD budget, requirements, and acquisition officials to consult 
each other and make trade-offs between cost, schedule, and performance 
early in the process; and 

• Crack down on the frequent changes to programs, which tend to cause 
many cost increases. 

While this bill is not intended as a panacea to cure all that ails the defense pro- 
curement process, it is an important next step in Congress’ continuing efforts to 
help DOD culturally reform the system. 

As I said a moment ago, for meaningful reform to truly endure, the commitment 
to reform must begin with the fiduciaries of the taxpayers’ dollars within the De- 
partment itself. 

Thank you, Mr. Chairman. I look forward to hearing from our witnesses today. 

Chairman Levin. Senator McCain, we thank you. 

Now we’ll call on our witnesses. First, we’ll call on Michael Sul- 
livan from the GAO. 

Would you please proceed? 

STATEMENT OF MICHAEL J. SULLIVAN, DIRECTOR, ACQUISI- 
TION AND SOURCING MANAGEMENT, GOVERNMENT AC- 
COUNTABILITY OFFICE 

Mr. Sullivan. Chairman Levin, Senator McCain, members of the 
committee. I’m pleased to be here today to discuss DOD’s acquisi- 
tion outcomes and the legislation proposed by this committee to im- 
prove them. I’ll make a brief oral statement and ask that my writ- 
ten testimony be placed in the record. 

Chairman Levin. It will be. 

Mr. Sullivan. We’ve been reporting for years on poor cost and 
schedule outcomes on DOD’s major weapon system acquisitions. As 
the chairman noted, most recently we reported that 95 programs 
in DOD’s current portfolio have grown in cost by $295 billion and 
are, on average, delivered about 21 months late. We believe there 
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are problems at the strategic and at the program levels that cause 
these outcomes. 

At the strategic level, DOD’s three systemic processes for build- 
ing its investment strategy are fragmented and broken. The re- 
quirement-setting process, known as the Joint Capabilities and In- 
tegration Development System (JCIDS) is stovepiped, it does not 
consider resources, and it approves nearly every proposal that it re- 
views. The funding process accepts programs with unrealistic cost 
estimates, and does not fully fund their development costs. These 
two processes are poorly integrated, and this poor communication 
leads to unhealthy competition, where too many programs are 
chasing too few dollars. 

Finally, at the program level, the acquisition process initiates 
programs with unreliable cost estimates and without knowledge 
from proper systems engineering analysis to understand each 
weapon system’s requirements and the resources that will be need- 
ed to achieve them. These programs move forward with too much 
technology, design, and manufacturing risk as a result. 

DOD understands this and has recently revised its policies to ad- 
dress some of these problems. Its new acquisition policy, for in- 
stance, encourages more systems engineering activity earlier in 
programs, competitive prototyping to gain knowledge more quickly 
and to maintain competition, earlier milestone reviews, and steer- 
ing boards to protect programs against the desire to add more re- 
quirements once they’ve started. 

Recent decisions by DOD on some programs have been encour- 
aging, and some of the newer programs appear to have undergone 
more disciplined reviews. 

For many years, there’s been a broad consensus that weapon sys- 
tem acquisition problems are serious and their resolution is over- 
due. With the Federal budget under increasing strain from the eco- 
nomic crisis facing our Nation, the time to change is now. 

In testimony before this committee last month. Secretary of De- 
fense Gates identified many of the systemic problems associated 
with acquisitions and indicated that efforts are underway to ad- 
dress them. 

We believe that the legislation this committee has proposed will 
help address the toughest problems, and we enthusiastically sup- 
port it. We believe it precisely targets key problem areas, provides 
much-needed oversight, and provides increased authority and inde- 
pendence to the critical functions of cost estimating and develop- 
ment testing by requiring them to report to the Secretary and to 
Congress. 

Among other things, its provision to require a full inventory of 
DOD’s current systems engineering skills is an excellent beginning 
to rebuilding that sorely-needed capability. Its addition of a termi- 
nation criterion for Nunn-McCurdy breaches sends a strong signal 
to programs to have realistic cost estimates when they start. 

It is important to state that there is also a need for changes to 
the overall acquisition culture and the incentives it provides. The 
culture should begin to change by resisting the urge to achieve the 
revolutionary, but unachievable, capability in one step by allowing 
technologies to mature in the technology base rather than forcing 
them on the acquisition programs too early, by ensuring that ur- 
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gent requirements are well-defined and quickly achievable, and by 
instituting shorter, more predictable development cycles. 

These changes will not be easy to make. Tough decisions must 
be made about DOD’s overall portfolio of weapon programs and 
about specific programs; and stakeholders from DOD, the military 
services industry, and Congress will have to play a constructive 
role in this decisionmaking. We see the proposed legislation dis- 
cussed here today as a very healthy step in that direction. 

Mr. Chairman, this concludes my statement. I’ll wait to answer 
questions. 

[The prepared statement of Mr. Sullivan follows:] 

Prepared Statement by Michael J. Sullivan 

Mr. Chairman and members of the committee: I am pleased to be here today to 
discuss the Department of Defense’s (DOD) management of its major weapon sys- 
tem acquisitions — an area that has been on the Government Accountability Office’s 
(GAO) high-risk list since 1990. Prior to and since that time, Congress and DOD 
have continually explored ways to improve acquisition outcomes without significant 
results. While the technological sophistication of DOD weapon systems is unparal- 
leled, major weapon programs continue to cost more, take longer to complete, and 
deliver fewer quantities and capabilities than originally planned. Last year we re- 
ported that the cumulative cost growth in DOD’s portfolio of 95 major defense acqui- 
sition programs was $295 billion from first estimates and the average delay in deliv- 
ering promised capabilities to the warfighter was 21 months. Clearly, some prob- 
lems are to be expected in developing weapon systems given the technical risks and 
complexities involved. However, all too often we have found that cost and schedule 
problems are rooted in poor planning, execution, and oversight. 

Investment in weapon systems is now at its highest level in two decades, and 
DOD plans to spend more than $357 billion over the next 5 years on major defense 
acquisition programs. Effective management of this substantial investment is crit- 
ical as competition for funding has increased dramatically within the department 
and across the government. DOD faces a number of fiscal pressures: the ongoing 
military campaigns in Afghanistan and Iraq, rising personnel costs, and the rebuild- 
ing and modernization of the force. In addition, the economic and fiscal crises now 
facing the Nation have required unprecedented spending by the Federal Govern- 
ment, and budget deficits are projected to remain high for many years to come. At 
a time when the Federal budget is strained by spending needs for a growing number 
of national priorities, it is imperative that DOD get the best value for every dollar 
it invests in weapon system programs. Every dollar wasted during the development 
and acquisition of weapon systems is money not available for other priorities within 
DOD and elsewhere in the government. 

Today, I will discuss: (1) the systemic problems that have contributed to cost, 
schedule, and performance problems in DOD’s acquisition of major weapon systems; 
(2) recent actions the department has taken to address these problems; (3) our ob- 
servations on the committee’s proposed acquisition reform legislation; and (4) steps 
that Congress and the department need to take to improve the future performance 
of acquisition programs. The statement includes findings from our July 2008 report 
on a knowledge-based funding approach and February 2009 report on potential 
changes to DOD’s acquisition management framework. ^ It also draws from our ex- 
tensive body of work on DOD’s acquisition of weapon systems. A list of our key prod- 
ucts is provided at the end of this statement. This work was conducted in accord- 
ance with generally accepted government auditing standards. Those standards re- 
quire that we plan and perform the audit to obtain sufficient, appropriate evidence 
to provide a reasonable basis for our findings and conclusions based on our audit 
objectives. We believe that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 


^GAO, Defense Acquisitions: A Knowledge-Based Funding Approach Could Improve Major 
Weapon System Program Outcomes, GAO-08-619 (Washington, DC: July 2, 2008), and Defense 
Acquisitions: Perspectives on Potential Changes to DOD’s Acquisition Management Framework, 
GAO-09-295R (Washington, DC: Feh. 27, 2009). 
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FRAGMENTED INVESTMENT DECISIONMAKING, UNEXECUTABLE PROGRAMS, AND LACK OF 
ACCOUNTABILITY UNDERLIE POOR ACQUISITION OUTCOMES 

Over the past several years our work has highlighted a number of underlying sys- 
temic causes for cost growth and schedule delays at hoth the strategic and program 
levels. At the strategic level, DOD’s processes for identifying warfighter needs, allo- 
cating resources, and developing and procuring weapon systems — which together de- 
fine DOD’s overall weapon system investment strategy — are fragmented. As a re- 
sult, DOD fails to effectively address joint warfighting needs and commits to more 
programs than it has resources for, thus creating unhealthy competition for funding. 
At the program level, a military Service typically establishes and DOD approves a 
business case containing requirements that are not fully understood and cost and 
schedule estimates that are based on overly optimistic assumptions rather than on 
sufficient knowledge. Once a program begins, it too often moves forward with inad- 
equate technology, design, testing, and manufacturing knowledge, making it impos- 
sible to successfully execute the program within established cost, schedule, and per- 
formance targets. Furthermore, DOD officials are rarely held accountable for poor 
decisions or poor program outcomes. 

DOD Lacks an Integrated Approach to Balance Weapon System Investments 

At the strategic level, DOD largely continues to define warfighting needs and 
make investment decisions on a Service-by-Service and individual platform basis, 
using fragmented decisionmaking processes. This approach makes it difficult for the 
department to achieve a balanced mix of weapon systems that are affordable and 
feasible and that provide the best military value to the joint warfighter. In contrast, 
we have found that successful commercial enterprises use an integrated portfolio 
management approach to focus early investment decisions on products collectively 
at the enterprise level and ensure that there is a sound basis to justify the commit- 
ment of resources.^ By following a disciplined, integrated process — during which the 
relative pros and cons of competing product proposals are assessed based on stra- 
tegic objectives, customer needs, and available resources, and where tough decisions 
about which investments to pursue and not to pursue are made — companies mini- 
mize duplication between business units, move away from organizational stovepipes, 
and effectively support each new development program they commit to. To be effec- 
tive, integrated portfolio management must have strong, committed leadership; em- 
powered portfolio managers; and accountability at all levels of the organization. 

DOD determines its capability needs through the Joint Capabilities and Integra- 
tion Development System (JCIDS). While JCIDS provides a framework for review- 
ing and validating needs, it does not adequately prioritize those needs from a joint, 
departmentwide perspective; lacks the agility to meet changing warfighter demands; 
and validates almost all of the capability proposals that are submitted. We recently 
reviewed JCIDS documentation related to new capability proposals and found that 
most — almost 70 percent — were sponsored by the military Services with little in- 
volvement from the joint community, including the combatant commands, which are 
responsible for planning and carrying out military operations.® Because DOD also 
lacks an anal 3 dic approach to determining the relative importance of the capabilities 
needed for joint warfighting, all proposals appear to be treated as equal priorities 
within the JCIDS process. By continuing to rely on capability needs denned pri- 
marily by the Services, DOD may be losing opportunities for improving joint 
warfighting capabilities and reducing the duplication of capabilities in some areas. 
The JCIDS process has also proven to be len^hy and cumbersome — taking on aver- 
age up to 10 months to validate a need — thus undermining the department’s efforts 
to effectively respond to the needs of the warfighter, especially those needs that are 
near term. Furthermore, the vast majority of capability proposals that enter the 
JCIDS process are validated or approved without accounting for the resources or 
technologies that will be needed to acquire the desired capabilities. Ultimately, the 
process produces more demand for new weapon system programs than available re- 
sources can support. 

The funding of proposed programs takes place through a separate process, the de- 
partment’s Planning, Programming, Budgeting, and Execution (PPBE) system, 
which is not synchronized with JCIDS. While JCIDS is a continuous, need-driven 
process that unfolds in response to capability proposals as they are submitted by 
sponsors, PPBE is a calendar-driven process comprising phases occurring over a 2- 


2 GAO, Best Practices: An Integrated Portfolio Management Approach to Weapon System In- 
vestments Could Improve DOD’s Acquisition Outcomes, GAO— 07-388 (Washington, DC: Mar. 30, 
2007). 

®GAO, Defense Acquisitions: DOD’s Requirements Determination Process Has Not Been Effec- 
tive in Prioritizing Joint Capabilities, GAO-08-1060 (Washington, DC: Sept. 25, 2008). 
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year cycle, which can lead to resource decisions for proposed programs that may 
occur several years later. In addition, because PPBE is structured by military Serv- 
ice and defense programs and not by the joint capability areas being used in JCIDS, 
it is difficult to link resources to capabilities. The PPBE process also largely allo- 
cates resources based on historical trends rather than on a strategic basis. Service 
shares of the overall budget have remained relatively static for decades, even 
though DOD’s strategic environment and warfighting needs have changed dramati- 
cally in recent years. Because DOD’s programming and budgeting reviews occur at 
the back end of the PPBE process — after the Services have developed their budg- 
ets — it is difficult and disruptive to make changes, such as terminating programs 
to pay for new, higher-priority programs. 

We recently reviewed the impact of the PPBE process on major defense acquisi- 
tion programs and found that the process does not produce an accurate picture of 
the department’s resource needs for weapon system programs, in large part because 
it allows too many programs to go forward with unreliable cost estimates and with- 
out a commitment to fully fund them."*^ The cost of many of the programs we re- 
viewed exceeded the funding levels planned for and reflected in the Future Years 
Defense Program (FYDP) — the department’s long-term investment strategy (see fig. 
1). DOD’s failure to balance its needs with available resources promotes an 
unhealthy competition for funding that encourages sponsors of weapon system pro- 
grams to pursue overly ambitious capabilities and underestimate costs to appear af- 
fordable. Rather than limit the number and size of programs or adjust require- 
ments, DOD opts to push the real costs of programs to the future. With too many 
programs underway for the available resources and high cost growth occurring in 
many programs, the department must make up for funding shortfalls by shifting 
funds from one program to pay for another, reducing system capabilities, cutting 
procurement quantities, or in rare cases terminating programs. Such actions not 
only create instability in DOD’s weapon system portfolio, they further obscure the 
true future costs of current commitments, making it difficult to make informed in- 
vestment decisions. 


“GAO-OS-eig. 
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Figure 1 : Funding Shortfalls at the Start of Development for Five Major Weapon 
System Programs' 
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"Multi-Mission Maritime Aircraft (MMA), Warfighter Information Network — Tactical (WIN-T), Future 
Combat Systems (FCS), Joint Strike Fighter (JSF), and Global Hawk. 


Initiating Programs with Inadequate Knowledge of Requirements and Resources 
Often Results in Poor Outcomes 

At the program level, the key cause of poor outcomes is the approval of programs 
with business cases that contain inadequate knowledge about requirements and the 
resources — funding, time, technologies, and people — needed to execute them. Our 
work in best practices has found that an executable business case for a program 
demonstrated evidence that: (1) the identified needs are real and necessary and that 
they can best be met with the chosen concept; and (2) the chosen concept can be 
developed and produced within existing resources. Over the past several years, we 
have found no evidence of the widespread adoption of such an approach for major 
acquisition programs in the department. Our annual assessments of major weapon 
systems have consistently found that the vast majority of programs began system 
development without mature technologies and moved into system demonstration 
without design stability. 

The chief reason for these problems is the encouragement within the acquisition 
environment of overly ambitious and lengthy product developments — sometimes re- 
ferred to as revolutionary or big bang acquisition programs — that embody too many 
technical unknowns and not enough knowledge about the performance and produc- 
tion risks they entail. The knowledge gaps are largely the result of a lack of early 
and disciplined systems engineering analysis of a weapon system’s requirements 
prior to beginning system development. Systems engineering translates customer 
needs into specific product requirements for which requisite technological, software, 
engineering, and production capabilities can be identified through requirements 
analysis, design, and testing. Early systems engineering provides the knowledge a 
product developer needs to identify and resolve performance and resource gaps be- 
fore product development begins by either reducing requirements, deferring them to 
the future, or increasing the estimated cost for the weapon system’s development. 
Because the government often does not perform the proper upfront requirements 
analysis to determine whether the program will meet its needs, significant contract 
cost increases can and do occur as the scope of the requirements changes or becomes 
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better understood by the government and contractor. Not only does DOD not con- 
duct disciplined systems engineering prior to the beginning of system development, 
it has allowed new requirements to be added well into the acquisition cycle. We 
have reported on the negative impact that poor systems engineering practices have 
had on several programs, such as the Global Hawk Unmanned Aircraft System, 
F-22A, Expeditionary Fighting Vehicle, and Joint Air-to-Surface Standoff Missile.® 

With high levels of uncertainty about requirements, technologies, and design, pro- 
gram cost estimates and related funding needs are often understated, effectively set- 
ting programs up for cost and schedule growth. We recently assessed the service and 
independent cost estimates for 20 major weapon system programs and found that 
while the independent estimates were somewhat higher, both estimates were too 
low in most cases.® In some of the programs we reviewed, cost estimates have been 
off by billions of dollars. For example, the initial service estimate for the develop- 
ment of the Marines’ Expeditionary Fighting Vehicle was about $1.1 billion. The de- 
partment’s Cost Analysis and Improvement Group (CAIG) estimated the develop- 
ment cost of the program to be $1.4 billion, but development costs for the program 
are now expected to be close to $3.6 billion. In the case of the Future Combat Sys- 
tem (FCS), the Army’s initial estimate for the development cost was about $20 bil- 
lion, while CAIG’s estimate was $27 billion. The department began the program 
using the program office’s estimate of $20 billion, but development costs for the FCS 
are now estimated to be $28 billion and the program is still dealing with significant 
technical risk. Estimates this far off the mark do not provide the necessary founda- 
tion for sufficient funding commitments and realistic long-term planning. 

The programs we reviewed frequently lacked the knowledge needed to develop re- 
alistic cost estimates. For example, program Cost Analysis Requirements Descrip- 
tion documents — used to build the program cost estimate — often lack sufficient de- 
tail about planned program content for developing sound cost estimates. Without 
this knowledge, cost estimators must rely heavily on parametric analysis and as- 
sumptions about system requirements, technologies, design maturity, and the time 
and funding needed. A cost estimate is then usually presented to decisionmakers as 
a single, or point, estimate that is expected to represent the most likely cost of the 
program but provides no information about the range of risk and uncertainty or 
level of confidence associated with the estimate. 

Lack of Accountability for Making Weapon System Decisions Hinders Achieving Suc- 
cessful Outcomes 

DOD’s requirements, resource allocation, and acquisition processes are led by dif- 
ferent organizations, thus making it difficult to hold any one person or organization 
accountable for saying no to a proposed program or for ensuring that the depart- 
ment’s portfolio of programs is balanced. DOD’s 2006 Defense Acquisition Perform- 
ance Assessment study observed that these processes are not connected organiza- 
tionally at any level below the Deputy Secretary of Defense and concluded that this 
weak structure induces instability and inhibits accountability. Furthermore, a 
former Under Secretary of Defense for Acquisitions, Technology, and Logistics has 
stated that weapon system investment decisions are a shared responsibility in the 
department and, therefore, no one individual is accountable for these decisions. Fre- 
quent turnover in leadership positions in the department exacerbates the problem. 
The average tenure, for example, of the Under Secretary of Defense for Acquisition, 
Technology, and Logistics over the past 22 years has been only about 20 months.’’ 

When DGD’s strategic processes fail to balance needs with resources and allow 
unsound, unexecutable programs to move forward, program managers cannot be 
held accountable when the programs they are handed already have a low probability 
of success. Program managers are also not empowered to make go or no-go deci- 
sions, have little control over funding, cannot veto new requirements, and have little 
authority over staffing. At the same time, program managers frequently change dur- 
ing a program’s development, making it difficult to hold them accountable for the 
business cases that they are entrusted to manage and deliver. 

The government’s lack of control over and accountability for decisionmaking is 
further complicated by DOD’s growing reliance on technical, business, and procure- 
ment expertise supplied by contractors. This reliance may reach the point where the 
foundation upon which decisions are based may be largely crafted by individuals 


®GAO, Best Practices: Increased Focus on Requirements and Oversight Needed to Improve 
DOD’s Acquisition Environment and Weapon System Quality, GAO— 08— 294 (Washington, DC: 
Feh. 1, 2008). 

«GAO-08-619. 

The position of Under Secretary of Defense for Acquisition was established in 1986 and the 
title was subsequently changed to the Under Secretary of Defense for Acquisition, Technology, 
and Logistics in 2000. Since 1986, there have been 11 Under Secretaries. 
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who are not employed by the government, who are not bound by the same rules gov- 
erning their conduct, and who are not required to disclose any financial or other 
personal interests they may have that conflict with the responsibilities they have 
performing contract tasks for DOD. For example, while the total planned commit- 
ments to major acquisition programs have doubled over recent years, the size of the 
department’s systems engineering workforce has remained relatively stable, leading 
program offices to rely more on contractors for systems engineering support. Fur- 
ther, in systems development, DOD typically uses cost-reimbursement contracts in 
which it generally pays the reasonable, allocable, and allowable costs incurred for 
the contractor’s best efforts, to the extent provided by the contract. The use of these 
contracts may contribute to the perpetuation of an acquisition environment that 
lacks incentives for contractors to follow best practices and keep costs and schedules 
in check. 

RECENT DOD POLICY CHANGES COULD IMPROVE FUTURE PERFORMANCE OF WEAPON 

SYSTEM PROGRAMS 

The department understands many of the problems that affect acquisition pro- 
grams and has recently taken steps to remedy them. It has revised its acquisition 
policy and introduced several initiatives based in part on direction from Congress 
and recommendations from GAO that could provide a foundation for establishing 
sound, knowledge-based business cases for individual acquisition programs. How- 
ever, to improve outcomes, DOD must ensure that its policy changes are consist- 
ently implemented and reflected in decisions on individual programs — not only new 
program starts but also ongoing programs as well. In the past, inconsistent imple- 
mentation of existing policy has hindered DOD’s efforts to execute acquisition pro- 
grams effectively. Moreover, while policy improvements are necessary, they may be 
insufficient unless the broader strategic issues associated with the department’s 
fragmented approach to managing its portfolio of weapon system investments are 
also addressed. 

In December 2008, DOD revised its policy governing major defense acquisition 
programs in ways intended to provide key department leaders with the knowledge 
needed to make informed decisions before a program starts and to maintain dis- 
ciplined development once it begins. The revised policy recommends the completion 
of key systems engineering activities before the start of development, includes a re- 
quirement for early prototyping, establishes review boards to identify and mitigate 
technical risks and evaluate the impact of potential requirements changes on ongo- 
ing programs, and incorporates program manager agreements to increase leadership 
stability and management accountability. The policy also establishes early milestone 
reviews for programs going through the pre — systems acquisition phase. In the past, 
DOD’s acquisition policy may have encouraged programs to rush into systems devel- 
opment without sufficient knowledge, in part, because no formal milestone reviews 
were required before system development. If implemented, these policy changes 
could help programs replace risk with knowledge, thereby increasing the chances of 
developing weapon systems within cost and schedule targets while meeting user 
needs. Some aspects of the policy were first pilot-tested on selected programs, such 
as the Joint Light Tactical Vehicle program, and indications are that these pro- 
grams are in the process of acquiring the requisite knowledge before the start of 
systems development. Some key elements of the department’s new acquisition policy 
include: 

• a new materiel development decision as a starting point for all programs 
regardless of where they are intended to enter the acquisition process, 

• a more robust Analysis of Alternatives (AOA) to assess potential materiel 
solutions that address a capability need validated through JCIDS, 

• a cost estimate for the proposed solution identified by the AOA, 

• early program support reviews by systems engineering teams, 

• competitive prototyping of the proposed system or key system elements 
as part of the technology development phase, 

• certifications for entry into the technology development and system devel- 
opment phases (as required by congressional legislation), 

• preliminary design review (PDR) that may be conducted before the start 
of systems development, and 

• configuration steering boards to review all requirements and technical 
changes that have potential to affect cost and schedule. 

As part of its strategy for enhancing the roles of program managers in major 
weapon system acquisitions, the department has established a policy that requires 
formal agreements among program managers, their acquisition executives, and the 
user community setting forth common program goals. These agreements are in- 
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tended to be binding and to detail the progress the program is expected to make 
during the year and the resources the program will be provided to reach these goals. 
DOD also requires program managers to sign tenure agreements so that their ten- 
ure will correspond to the next major milestone review closest to 4 years. The de- 
partment acknowledges that any actions taken to improve accountability must be 
based on a foundation whereby program managers can launch and manage pro- 
grams toward successful performance, rather than focusing on maintaining support 
and funding for individual programs. DOD acquisition leaders have also stated that 
any improvements to program managers’ performance depend on the department’s 
ability to promote requirements and resource stability over weapon system invest- 
ments. 

Over the past few years, DOD has also been testing portfolio management ap- 
proaches in selected capability areas — command and control, net-centric operations, 
battlespace awareness, and logistics — to facilitate more strategic choices for resource 
allocation across programs. The department recently formalized the concept of capa- 
bility portfolio management, issuing a directive in 2008 that established policy and 
assigned responsibilities for portfolio management. The directive established nine 
joint capability area portfolios, each to be managed by civilian and military co-leads. 
While the portfolios have no independent decisionmaking authority over require- 
ments determination and resource allocation, according to some DOD officials, they 
provided key input and recommendations in this year’s budget process. However, 
without portfolios in which managers have authority and control over resources, the 
department is at risk of continuing to develop and acquire systems in a stovepiped 
manner and of not knowing if its systems are being developed within available re- 
sources. 


OBSERVATIONS ON PROPOSED ACQUISITION REFORM LEGISLATION 

Overall, we believe that the legislative initiatives being proposed by the com- 
mittee have the potential, if implemented, to lead to significant improvements in 
DOD’s management of weapon system programs. Several of the initiatives — includ- 
ing the increased emphasis on systems engineering and developmental testing, the 
requirement for earlier PDRs, and the strengthening of independent cost estimates 
and technology readiness assessments — could instill more discipline into the front 
end of the acquisition process when it is critical for programs to gain knowledge. 
Establishing a termination criterion for Nunn-McCurdy cost breaches could help 
prevent the acceptance of unrealistic cost estimates as a foundation for starting pro- 
grams.® Having greater involvement by the combatant commands in determining re- 
quirements and requiring greater consultation between the requirements, budget, 
and acquisition processes could help improve the department’s efforts to balance its 
portfolio of weapon system programs. In addition, several of the proposals as cur- 
rently drafted will codify what DOD policy already calls for, but are not being imple- 
mented consistently in weapon programs. 

Section 101: Systems Engineering Capabilities 

Requires DOD to: (1) assess the extent to which the department has in place the 
systems engineering capabilities needed to ensure that key acquisition decisions are 
supported by a rigorous systems analysis and systems engineering process; and (2) 
establish organizations and develop skilled employees to fill any gaps in such capa- 
bilities. 

The lack of disciplined systems engineering analysis conducted prior to starting 
system development has been a key factor contributing to poor acquisition outcomes. 
Systems engineering activities — requirements analysis, design, and testing — are 
needed to ensure that a weapon system program’s requirements are achievable and 
designable given available resources, such as technologies. In recent years, DOD has 
taken steps to improve its systems engineering capabilities by establishing a Sys- 
tems and Software Engineering Center of Excellence and publishing guidance to as- 
sist the acquisition workforce in the development of systems engineering plans, edu- 
cation, and training. However, as the National Research Council recently reported, 
DOD’s systems engineering capabilities have declined over time and shifted increas- 
ingly to outside contractors.® A comprehensive assessment to determine what sys- 


® 10 U.S.C. §2433fa)(5) requires the Secretary of Defense to report to Congress when a pro- 
gram’s acquisition unit cost increases by at least 25 percent over the current baseline estimate 
or increases by at least 50 percent over the original baseline estimate. 

® National Research Council, Pre-Milestone A and Early-Phase Systems Engineering: A Retro- 
spective Review and Benefits for Future Air Force Systems Acquisition (Washington, DC: Feb- 
ruary 2008). 
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terns engineering capabilities are in place and what capabilities are needed, as rec- 
ommended in the proposed legislation, is a critical first step in enhancing the func- 
tion of systems engineering in DOD acquisitions. At the same time, it will be impor- 
tant for DOD to implement steps to ensure systems engineering is applied in the 
right way and at the right time. 

Section 102: Developmental Testing 

Requires DOD to reestablish the position of Director of Developmental Test and 
Evaluation and requires the Services to assess and address any shortcomings in 
their developmental testing organizations and personnel. 

Robust developmental testing efforts are an integral part of the systems develop- 
ment process. They help to identify, evaluate, and reduce technical risks, and indi- 
cate whether the design solution is on track to satisfy the desired capabilities. As 
the Defense Science Board reported in 2008, developmental testing in weapon sys- 
tem programs needs to be improved.^^ We believe that developmental testing would 
be strengthened by a formal elevation of its role in the acquisition process and the 
reestablishment of a Director of Developmental Test and Evaluation position. Fur- 
thermore, requiring the Director to prepare an annual report for Congress summa- 
rizing DOD’s developmental test and evaluation activities would provide more ac- 
countability. We also agree that the military Services should be required to assess 
their respective developmental testing entities and address any shortcomings. This 
action would help ensure that the Services have the knowledge and capacity for ef- 
fective developmental test efforts. 

Section 103: Technical Maturity Assessments 

Makes it the responsibility of the Director of Defense Research and Engineering 
(DDR&E) to periodically review and assess the technological maturity of critical 
technologies used in major defense acquisition programs. 

Ensuring that programs have mature technology before starting systems develop- 
ment is critical to avoiding cost and schedule problems, yet for many years we have 
reported that a majority of programs go forward with immature technologies and 
experience significant cost growth. Legislation enacted by Congress in 2006, requir- 
ing DOD to certify that the technology in a program has been demonstrated in a 
relevant environment before it receives approval to start system development, has 
begun to help address this problem. Since the legislation was enacted, DOD has 
asked the DDR&E to conduct independent reviews of technology readiness assess- 
ments for system development milestone decisions. Although DDR&E reviews are 
advisory in nature, we have seen reviews that have pushed programs to do more 
to demonstrate technology maturity. The improvements that this proposed legisla- 
tion, as currently written, is intended to bring about may already be occurring in 
DOD. Congress, however, may wish to consider requiring the DDR&E to conduct 
technology readiness reviews not just periodically, but for all major defense acquisi- 
tion programs, and whether or not DDR&E has the capacity and resources to effec- 
tively conduct technology assessments. 

Section 104: Independent Cost Assessment 

Establish a Director of Independent Cost Assessment to ensure that cost esti- 
mates for major defense acquisition programs are fair, reliable, and unbiased. 

Within DOD, the CAIG is the organization responsible for conducting independent 
costs estimates for major defense acquisition programs. The CAIG reports to the de- 
partment’s Director of Program Analysis and Evaluation, but its principal customer 
is the Under Secretary of Defense for Acquisition, Technology, and Logistics. We be- 
lieve that establishing an independent assessment office that reports directly to the 
Secretary or Deputy Secretary of Defense and to Congress — similar to the Office of 
the Director of Operation Test and Evaluation — would more fully integrate cost esti- 
mating with the acquisition management framework and provide an increased level 
of accountability. We see no reason why CAIG should not form the basis of the pro- 
posed organization. Congress may also wish to consider appointing the Director for 
a time-certain term and making the Director responsible for prescribing cost-esti- 
mating policy and guidance and for preparing an annual report summarizing cost 
estimates for major acquisition programs. Ultimately, however, improved cost esti- 
mating will only occur if there is a better foundation for planning and acquiring 
weapon system programs — one that promotes well-defined requirements, is knowl- 
edge-based and informed by disciplined systems engineering, requires mature tech- 
nology, and adheres to shorter development cycle times. 


u’ Defense Science Board, Report on Developmental Test & Evaluation (Washington, DC: May 
2008 ). 
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Section 105: Role of Combatant Commanders 

Requires the Joint Requirements Oversight Council (JROC) to seek and consider 
input from the commanders of the combatant commands in identifying joint military 
requirements. 

Requirements determination in DOD, particularly for major weapon system pro- 
grams, continues to be driven largely by the military Services. Studies by the De- 
fense Science Board, Center for Strategic and International Studies, and others have 
revealed that although the combatant commands — which are responsible for plan- 
ning and executing military missions — are the principal joint warfighting customer 
in DOD, they have played a limited role in determining requirements. Currently, 
the JROC is doing more to seek out and consider input from the combatant com- 
mands through regular trips and meetings to discuss capability needs and 
resourcing issues. However, many of the combatant commands do not believe that 
their needs, which are reflected through the Integrated Priority List process, are 
sufficiently addressed through the department’s JCIDS process. For the combatant 
commands to meet this proposed legislative mandate and have more influence in es- 
tablishing requirements, DOD should consider providing the combatant commands 
with additional resources to establish robust anal 3 dical capabilities for identifying 
and assessing their capability needs. Ultimately, the department must better 
prioritize and balance the needs of the military Services, combatant commands, and 
other defense components, and be more agile in responding to near-term capability 
needs. 

Section 201: Trade-offs of Cost, Schedule, and Performance 

Requires consultation between the budget, requirements, and acquisition proc- 
esses to ensure the consideration of trade-offs between cost, schedule, and perform- 
ance early in the process of developing major weapon systems. 

As currently structured, DOD’s budget, requirements, and acquisition processes 
do not operate in an integrated manner. The function and timing of the processes 
are not sufficiently synchronized, and the decisionmakers for each process are moti- 
vated by different incentives. These weaknesses have contributed to the develop- 
ment of a portfolio with more programs than available resources can support and 
programs that launch into system development without executable business cases. 
We have recommended that the department establish an enterprisewide portfolio 
management approach to weapon system investment decisions that integrates the 
determination of joint warfighting needs with the allocation of resources, and cuts 
across the Services by functional or capability area.^i To ensure the success of such 
an approach, we believe that the department should establish a single point of ac- 
countability with the authority, responsibility, and tools to implement portfolio man- 
agement effectively. 

Section 202: Preliminary Design Review 

Require the completion of a PDR and a formal post-PDR assessment before a 
major defense acquisition program receives approval to start system development. 

We have found that a key deliverable in a knowledge-based acquisition process 
is the preliminary design of the proposed solution based on a robust systems engi- 
neering assessment prior to making a large financial commitment to system devel- 
opment. Early systems engineering provides the knowledge needed by a developer 
to identify and resolve gaps, such as overly optimistic requirements that cannot be 
met with current resources, before product development begins. Consequently, DOD 
would have more confidence that a particular system could successfully proceed into 
a detailed system development phase and meet stated performance requirements 
within cost, schedule, risk, and other relevant constraints. The recently revised 
DOD acquisition policy places an increased emphasis on programs planning for PDR 
prior to the start of system development but does not go as far as making it a re- 
quirement to do so. We support any effort to add controls to the acquisition process 
to ensure that timely and robust systems engineering is conducted before major in- 
vestment decisions, such as the approval to start system development, are made. 

Section 203: Life-Cycle Competition 

Require DOD to adopt measures recommended by the 2008 Defense Science Board 
Task Force on Defense Industrial Structure for Transformation — such as competi- 
tive prototyping, dual sourcing, open architectures, periodic competitions for sub- 
system upgrades, and licensing of additional suppliers — to maximize competition 
throughout the life of a program. 


iiGAO-07-388. 
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We have reported in the past on the problem of diminishing competition and the 
potential benefits of more competition, In discussing the environment that leads 
to poor acquisition outcomes, we have noted that changes within the defense sup- 
plier base have added pressure to this environment. We noted that in 2006, a DOD- 
commissioned study found that the number of fully competent prime contractors 
competing for programs had fallen from more than 20 in 1985 to only 6, and that 
this has limited DOD’s ability to maximize competition in order to reduce costs and 
encourage innovation. However, avenues exist for reducing costs through competi- 
tion. For example, we reported that although continuing an alternate engine pro- 
gram for the Joint Strike Fighter would cost significantly more in development costs 
than a sole-source program, it could, in the long run, reduce overall life-cycle costs 
and bring other benefits. 

Section 204: Nunn-McCurdy Breaches 

Requires that a major defense acquisition program that experiences a critical 
Nunn-McCurdy cost breach be terminated unless the Secretary of Defense certifies 
that: (1) continuing the program is essential to national security and the program 
can be modified to proceed in a cost-effective manner; and (2) the program receives 
a new milestone approval prior to the award of any new or modified contract extend- 
ing the scope of the program. 

In order for DOD to improve its program outcomes, realistic cost estimates must 
be required when programs are approved for development initiation. DOD often 
underestimates costs in large part because of a lack of knowledge and overly opti- 
mistic assumptions about requirements and critical technologies. This underesti- 
mation is also influenced by DOD’s continuing failure to balance its needs with 
available resources, which promotes unhealthy competition among programs and en- 
courages programs to overpromise on performance capabilities and underestimate 
cost. This false optimism is reinforced by an acquisition environment in which there 
are few ramifications for cost growth and delays. Only in very rare instances have 
programs been terminated for poor performance. When DOD consistently allows un- 
sound, unexecutable programs to begin with few negative ramifications for poor out- 
comes, accountability suffers. As section 204 proposes, the strengthening of the 
Nunn-McCurdy provision — by including the potential termination of programs that 
experience critical cost growth — could facilitate a change in DOD’s behavior by pre- 
venting the acceptance of unrealistic cost estimates as a foundation for program ini- 
tiation and placing more accountability on senior DOD leadership for justifying pro- 
gram continuation. Programs may thus be forced to be more candid and upfront 
about potential costs, risks, and funding needs, and the likelihood of delivering a 
successful capability to the warfighter at the cost and in the time promised may 
grow. 

Section 205: Organizational Conflicts of Interest 

Prohibits systems engineering contractors from participating in the development 
or construction of major weapon systems on which they are advising DOD, and re- 
quires tightened oversight of organizational conflicts of interest by contractors in the 
acquisition of major weapon systems. 

The defense industry has undergone significant consolidation in recent years 
which has resulted in a few large, vertically integrated prime contractors. This con- 
solidation creates the potential for organizational conflicts of interest where, for ex- 
ample, one business unit of a large company may be asked to provide systems engi- 
neering work on a system being produced by another unit of the same company. As 
the Defense Science Board has recognized, these conflicts of interest may lead to im- 
paired objectivity, which may not be mitigated effectively through techniques such 
as erecting a firewall between the employees of the two units. While the Federal 
Acquisition Regulation currently covers some cases of potential organizational con- 
flicts of interest involving the systems engineering function, there may be a need 
for additional coverage in this area. In general, we would support efforts to enhance 
the oversight of potential organizational conflicts of interest, particularly in the cur- 
rent environment of a heavily consolidated defense industry. 

Section 206: Acquisition Excellence 

Establishes an annual awards program to recognize individuals and teams that 
make significant contributions to the improved cost, schedule, and performance of 
defense acquisition programs. 


^^GAO, Defense Acquisitions: Better Weapon Program Outcomes Require Discipline, Account- 
ability, and Fundamental Changes in the Acquisition Environment, GAO— 08— 782T (Washington, 
DC: June 3, 2008). 
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We support the creation of an annual awards program to recognize individuals 
and teams for improving the cost, schedule, and performance of defense acquisition 
programs. We have reported that meaningful and lasting reform will not be 
achieved until the right incentives are established and accountability is bolstered at 
all levels of the acquisition process. The need for incentives emerged as a significant 
issue in our recent discussions with acquisition experts examining potential changes 
to the acquisition processes enumerated in last year’s defense authorization act. The 
discussions revealed that those changes may not achieve the desired improvement 
in acquisition outcomes unless they are accompanied by changes in the overall ac- 
quisition environment and culture, and the incentives they provide for success. 

CONCLUDING OBSERVATIONS ON WHAT REMAINS TO BE DONE 

A broad consensus exists that weapon system problems are serious and that their 
resolution is overdue. With the Federal budget under increasing strain from the Na- 
tion’s economic crisis, the time for change is now. DOD is off to a good start with 
the recent revisions to its acquisition policy, which, if implemented properly, should 
provide a foundation for establishing sound, knowledge-based business cases before 
launching into development and for maintaining discipline after initiation. The new 
policy will not work effectively, however, without changes to the overall acquisition 
environment. Resisting the urge to achieve the revolutionary but unachievable capa- 
bility, allowing technologies to mature in the science and technology base before 
bringing them onto programs, ensuring that requirements are well-defined and do- 
able, and instituting shorter development cycles would all make it easier to estimate 
costs accurately, and then predict funding needs and allocate resources effectively. 
But these measures will succeed only if the department uses an incremental ap- 
proach. Constraining development cycle times to 5 or 6 years will force more man- 
ageable commitments, make costs and schedules more predictable, and facilitate the 
delivery of capabilities in a timely manner. 

Acquisition problems are also likely to continue until DOD’s approach to man- 
aging its weapon system portfolio: (1) prioritizes needs with available resources, 
thus eliminating unhealthy competition for funding and the incentives for making 
programs look affordable when they are not; (2) facilitates better decisions about 
which programs to pursue and which not to pursue given existing and expected 
funding; and (3) balances the near-term needs of the joint warfighter with the long- 
term need to modernize the force. Achieving this affordable portfolio will require 
strong leadership and accountability. Establishing a single point of accountability 
could help the department align competing needs with available resources. 

The department has tough decisions to make about its weapon systems and port- 
folio, and stakeholders, including military Services, industry, and Congress, have to 
play a constructive role in the process toward change. Reform will not be achieved 
until DOD changes its acquisition environment and the incentives that drive the be- 
havior of its decisionmakers, the military Services, program managers, and the de- 
fense industry. 

Mr. Chairman, this concludes my prepared statement. I would be happy to answer 
any questions you may have at this time. 

CONTACTS AND ACKNOWLEDGEMENTS 

For further information about this statement, please contact Michael J. Sullivan 
(202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congres- 
sional Relations and Public Affairs may be found on the last page of this statement. 
Individuals who made key contributions to this statement include John Oppenheim, 
Charlie Shivers, Dayna Foster, Matt Lea, Susan Neill, Ron Schwenn, and Bruce 
Thomas. 

Chairman Levin. Thank you so much, Mr. Sullivan. 

Dr. Gansler? 

STATEMENT OF HON. JACQUES S. GANSLER, CHAIRMAN, DE- 
FENSE SCIENCE BOARD TASK FORCE ON INDUSTRIAL 
STRUCTURE FOR TRANSFORMATION 

Dr. Gansler. Mr. Chairman, members of the committee, thank 
you very much for this honor of appearing before you at what I 
think is a critical period and on such an important topic. 

I don’t have to tell this committee of the incredible national secu- 
rity challenges that the United States is facing in the 21st century. 
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brought on by the rather dramatic world changes that I believe re- 
quire a new, holistic view of security — DOD, Department of State, 
Department of Homeland Security, Director of National Intel- 
ligence, and so forth — and utilizing both hard and soft power, and 
addressing a very broad spectrum of the security missions with 
great unpredictability and covering the full spectrum, from ter- 
rorism all the way through nuclear deterrence. 

I would also emphasize that we need to take full advantage of 
globalization of the technology of industry, not restricting or gam- 
ing the benefits from globalization through restrictive legislation. 

In recognizing the long-term national security implications of the 
global financial crisis, the need for energy security, worldwide 
pandemics, the impact of climate change, growing anti- 
globalization backlash, and the challenging U.S. demographics. We 
have to do all of this, as Senator McCain emphasized, in a likely 
fiscally-constrained budget environment. 

Now, to address these challenges, I believe four highly inter- 
related acquisition issues must be addressed, and they have to be 
addressed by both DOD and Congress. First, what goods and serv- 
ices to buy; that’s the requirements process. Second, how to buy 
them; that’s the acquisition reform. Third, who does the acquiring; 
we have major issues in the acquisition workforce. Fourth, from 
whom it is acquired; namely, the industrial base. 

Now, I wish I could tell you that there is some silver bullet to 
address all of these needed changes, but it truly requires a very 
broad set of initiatives in each of the four areas if the Nation is 
to achieve the required 21st century national security posture. 

In my prepared testimony, which, Mr. Chairman, I would appre- 
ciate being put into the record 

Chairman Levin. It will be. 

Dr. Gansler. 1 listed the required actions in each of these 

four areas, and I’d be pleased to discuss any of these with you at 
any time. However, for now let me summarize. 

I believe this is a very critical period, perhaps somewhat similar 
to the period following the launch of Sputnik or the fall of the Ber- 
lin Wall. Today, the security world is changing dramatically, espe- 
cially since September 11 — geopolitically, technologically, threats, 
missions, warfighting, commercially, et cetera — and this holistic 
perspective that I mentioned is required. Moreover, a decade of 
solid budget growth, which I believe will almost certainly change, 
has deferred the difficult choices — for example, between more 20th 
century equipment versus 21st century equipment — and the con- 
trolling acquisition policies, practices, laws, and so forth, as well as 
the Services’ budgets and requirements priorities have not been 
transformed sufficiently to match the needs of this new world. In 
fact, there’s still an emphasis on resetting versus modernization. 

Now, leadership is required to achieve the needed changes. You 
look at the literature on culture change, which I think this clearly 
is, two things are required to successfully bring about the needed 
changes. First is the recognition of the need, a crisis. In this case, 
I believe it is a combination of the economic — the budget, if you 
will, crisis — and the changing security needs, along with the short- 
age of the senior acquisition experienced personnel to address these 
needs. Second, leadership, with a vision, a strategy, and an action 
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plan. I honestly believe that President Obama, Congress, and Sec- 
retary Gates support the needed changes; however, it’s pretty clear 
that the changes can be expected to be severely resisted. Signifi- 
cant change always is. 

I would start, as my highest priority, with the important role of 
the service chiefs and secretaries in recognizing and promoting sen- 
ior acquisition personnel, military and civilian. Over the last dec- 
ade, the DOD acquisition workforce has been greatly undervalued. 
DOD leadership now must demonstrate their personal recognition 
of the critical nature of senior experienced acquisition personnel 
and of the smart acquisition practices that they would bring to 
America’s military posture in the 21st century. 

As my second priority, I would emphasize the importance of 
weapons costs as a true military requirement, to achieve adequate 
numbers of weapons in a resource-constrained environment. This 
will require enhanced systems engineering, including cost-perform- 
ance tradeoffs, throughout both the government and industry, and 
incentives to industry for achieving lower cost. 

By the way, this has been done before; for example, with the 
Joint Direct Attack Munition missile, where the Air Force Chief of 
Staff said it should hit the target and cost under $40,000 each. It 
now sells for under $20,000 and precisely hits the targets. 

Finally, as my third priority, I would emphasize the value of 
rapid acquisition, from both its military and its economic benefits, 
which will require the full use of spiral development, with each 
block based on proven, tested technology and continuous user and 
logistician feedback for the subsequent block improvements, and 
with the option of continuous effective competition, at the prime or 
at the sublevel. If they’re not continuously achieving improved per- 
formance at lower costs, then they should be competed. 

Achieving these required changes will take political courage and 
sustained, strong leadership by both the executive and legislative 
branches, working together. I hope, and firmly believe, that it can 
be achieved. The American public, and particularly our fighting 
men and women, deserve it, and the Nation’s future security de- 
pends upon it. 

Thank you. 

[The prepared statement of Dr. Gansler follows:] 

Prepared Statement by Hon. Jacques S. Gansler i 

Mr. Chairman, members of the committee, thank you for the honor of appearing 
before you at this critical period, and on such an important topic. 

I need not tell you that the U.S., in the 21st century, faces incredible national 
security challenges — brought on by dramatic world changes that require: 

• A new. Holistic View of Security (e.g., DOD, State, DHS, DNI, etc.) — utilizing 
both “hard” and “soft” power 

• Addressing a Broad Spectrum of Security Missions — with great unpredict- 
ability (from Terrorism to Nuclear Deterrence) 

• Taking full advantage of Globalization (of Technology, Industry, etc.) 

• Recognizing the long-term national security implications of: 

• The global financial crisis 

• The need for energy security 

• Worldwide pandemics 


^Dr. Gansler is Professor and Roger C. Lipitz Chair, as well as Director of the Center for Pub- 
lic Policy and Private Enterprise at the School of Public Policy, University of Maryland. He 
served as Under Secretary of Defense (Acquisition, Technology, and Logistics) from 1997-2001. 
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• The impact of climate change 

• The growing anti-globalization backlash 

• The challenging U.S. demographics 

• To do all of this in a likely fiscally-constrained budget environment 

To address these challenges, four, highly-interrelated acquisition issues must be 
addressed (by the DOD and Congress): 

• What goods and services to buy (the “requirements” process) 

• How to buy them (“acquisition reform”) 

• Who does the acquiring (the acquisition workforce) 

• From whom is it acquired (the industrial base) 

I wish I could tell you that there was a “silver bullet” to address the needed 
changes; but this truly requires a broad set of initiatives in each of the four areas — 
if the Nation is to achieve the required 21st century national security posture. 

This need, for the four sets of broad changes, was emphasized in a recent Defense 
Science Board report; where they found: 

• “DOD policies, processes, and management of the Defense Acquisition 
Enterprise (broadly defined) impede the transition to an effective, agile, and 
affordable overall, joint military force for the 21st century.” 

• “U.S. Government policies, practices, and processes do not facilitate the 
development, deployment, and support of tbe innovative, affordable, and 
rapidly acquired weapons, systems, and services needed for the 21st cen- 
tury forces.” 

• “The absence of many of the needed skills, (e.g., experienced program 
management, systems engineering, biotech, advanced IT) in DOD’s acquisi- 
tion workforce, (particularly at the senior military and civilian levels), com- 
bined with the coming retirement and prior, large acquisition workforce re- 
ductions, significantly impedes the development, production, support, and 
oversight of the military capabilities needed for tbe 21st century.” 

• “Government acquisition policies and industry trends (e.g., further hori- 
zontal and vertical consolidations) will not produce the required competi- 
tive, responsive, efficient, and innovative National Security Industrial 
Base.” 

So let me (very briefly) summarize the changes required in each of the four, crit- 
ical (and interrelated) areas: [in priority order within each category] 

What is acquired: 

To meet the wide range of challenges, within a resource-constrained environment, 
the Nation must focus on: 

1. Lower cost systems and services 

2. Optimized, net-centric systems-of-systems (vs. individual “platforms”) 

3. A “reserve” of resources to rapidly respond to combat commanders’ ur- 
gent needs 

4. More “balanced” allocation of resources (to address “irregular” oper- 
ations): C^ISR, unmanned systems. Special Forces, “Land Warriors,” 
cyberdefense, etc; [and these resources must be moved from the 
Supplementals into tbe base budget] 

5. Interoperability of “joint” systems; and coalition systems 

6. Planning, equipping, and exercising “as we’ll figbt”: witb allies, multi- 
agencies, and “contractors on tbe battlefield” 

How goods and services are acquired: 

To achieve higher performance at lower costs and faster: 

1. Require “cost” as a design/military “requirement” (because cost, in a re- 
source-constrained environment, is numbers) 

2. Provide viable, continuous “competition options” (as the incentive for 
higher performance at lower costs) e.g. competitive prototypes, competitive 
split-buys, etc. 

3. Fully utilize “spiral development,” with demonstrated technologies (be- 
cause it is lower cost, lower risk, faster to field; maintains the option of 
competition; avoids obsolescence; can respond rapidly to combat needs) 

4. Make maximum use of commercial products and services (at all lev- 
els — utilizing Other Transactions Authority; especially at lower tiers) 

5. Institutionalize a “Rapid acquisition,” parallel process (to respond to 
COCOM urgent needs) 
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6. Create incentives for contractors to achieve desired results (in cost, 
schedule, and performance) 

7. Implement modern, integrated, enterprise-wide IT systems (logistics, 
business, personnel, etc.) — including linking Government and Industry 

8. Address Conflict of Interest concerns (from LSI^Make/Buy^SETA); 
but don’t reduce the value of relevant experience 

Who does the aequiring: 

A flexible, responsive, efficient, and effective acquisition program (for sophisti- 
cated, hi-tech goods and services) requires “smart buyers.” 

This depends on both quantity and quality of senior and experienced military and 
civilian personnel (especially for expeditionary operations). In the last decade-plus, 
this “requirement” has not been met! In fact, the acquisition workforce declined on 
seniority and quantity even as procurement appropriations increased. 
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Therefore, one of the Nation’s highest priorities (not just in the DOD) must be 
to address the acquisition workforce. 

• The DOD, especially, has an acquisition workforce problem: 

• Greatly reduced senior officers and SESs 

• In 1990 the Army had five general officers with contract background; 
in 2007 they had zero. 

• In 1995, the Air Force had 40 General Officers in Acquisition, today 
24; and in 1995, 87 SESs and today 49 

• The Defense Contract Management Agency (25,000 people in 1990 
down to 10,000 today; and 4 General Officers to 0) 

These reductions (due to the under valuing of the importance of the acquisition 
workforce) introduce “opportunities” for “waste, fraud and abuse” (e.g., 90 fraud 
cases under review from war zone; examples of poor acquisition process results, such 
as the Air Force Tanker, the Presidential Helicopter, etc.). These Government acqui- 
sition workforce issues must be addressed. I believe that President Obama, Con- 
gress, and Secretary Gates all agree on this (but it will take the priority attention 
of the Service Chiefs and Secretaries to make it a priority). 

From whom goods and services are acquired: 

To quote, again, from the recent Defense Science Board study (on the desired 21st 
century defense industry): 
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“The last two decades have seen a consolidation of the Defense Industry 
around 20th Century Needs — The next step is DOD leadership in trans- 
forming to a 21st Century National Security Industrial Structure.” 

The “vision” for this 21st Century National Security Industrial Base (which appro- 
priate government actions, i.e. acquisitions, policies, practices, and laws, must 
incentivize and facilitate) are: 

1. Efficient, responsive, technologically advanced, highly-competitive (at 
all levels, including public and private sectors) 

2. Globalized (utilizing “best in class”) — requires significant changes to 
U.S. export controls (i.e., changes to ITAR, EAR, etc.) 

3. Healthy (profitable); and investing in IR&D and capital equipment 
(rules should separate IR&D and B&P) 

4. Includes commercial firms and equipment, and meiximizes dual-use fa- 
cilities and workforce (barriers must be removed) 

5. “Independent” systems-of-systems architecture and systems engineer- 
ing firms (to support the Government — as the integrator) 

6. Merger and Acquisition policy guidelines to be based on this vision 

7. Strong Government-Industry Communications encouraged 

8. All non-inherently-governmental work to be done competitively (public 
vs. private, for current government work) 

9. Structural changes to eliminate appearance, or reality, of conflict of in- 
terest (regarding “vertical integration”) — but great care to assure relevant — 
experienced firms and people involved 

In summary, I believe this is a critical period, perhaps similar to the period fol- 
lowing the launch of Sputnik or the fall of the Berlin Wall. Today the security world 
is changing dramatically — especially since September 11, 2001 (geopolitically, tech- 
nologically, threats, missions, warfighting, commercially, etc.) — and a holistic per- 
spective is required (including State, DHS, and DNI, as well as coalition operations). 
Moreover, a decade of solid budget growth — which will almost certainly change — has 
deferred difficult choices (between more 20th century equipment vs. 21st century 
equipment). The controlling acquisition policies, practices, laws, etc. and the Serv- 
ices’ budgets and “requirements” priorities have not been transformed sufficiently 
to match the needs of this new world (in fact, there is still an emphasis on “reset- 
ting” vs. “modernization”). 

Leadership is required to achieve the needed changes! All of the literature on “cul- 
tural change” (which this clearly must be) state that two things are required to suc- 
cessfully bring about the needed changes: 

• Recognition of the need (a “crisis”) 

In this case, I believe it is the combination of economic/budget “crisis,” the chang- 
ing security needs, and the shortage of the senior acquisition-experienced personnel 
to address the needs; and 

• Leadership — with a “vision,” a “strategy,” and an “action plan.” I believe 
that President Obama, Congress, and Secretary Gates support the needed 
changes. However, the changes can be expected to be severely resisted — sig- 
nificant change always is! 

I would start with the important role of the Service Chiefs and Secretaries in rec- 
ognizing, and promoting senior acquisition personnel (military and civilian) in order 
to demonstrate their personal recognition of the critical nature of smart acquisition 
practices to American’s military posture in the 21st century. As my second priority, 
I would emphasize the importance of weapons costs as a military requirement (to 
achieve adequate members of weapons in a resource-constrained environment) — 
which will require enhanced systems engineering (throughout both government and 
industry) and incentives to industry for achieving lower cost systems. Einally, as my 
third priority, I would emphasize the value of “rapid acquisition”, for both its mili- 
tary and economic benefits — which will require the full use of “spiral development” 
(with each “block” based on proven/tested technology, and continuous user and logis- 
tician feedback, for subsequent “block” improvement — and with the option of effec- 
tive competition (at the prime and/or sub-level, if they are not continuously achiev- 
ing improved performances at lower and lower costs). 

Achieving these required changes will take political courage and sustained, strong 
leadership — by both the executive and legislative branches (working together). I 
hope, and firmly believe, it can be achieved. The American public, and particularly, 
our fighting men and women, deserve it — and the Nation’s future security depends 
upon it. 

Thank you. 
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Chairman Levin. Dr. Gansler, thank you so much. 

Dr. Kaminski? 

STATEMENT OF HON. PAUL G. KAMINSKI, CHAIR, COMMITTEE 

ON PRE-MILESTONE A SYSTEMS ENGINEERING, AIR FORCE 

STUDIES BOARD, NATIONAL RESEARCH COUNCIL 

Dr. Kaminski. Mr. Chairman, Senator McCain, and members of 
the committee, first of all, I want to thank you for your leadership 
on these critical acquisition issues and for the invitation to testify. 

Since you’ve asked me to testify, first, in my role as Chairman 
of the National Research Council’s Study on Pre-Milestone A Sys- 
tems Engineering, with your permission I would ask that my state- 
ment, which includes a full summary, be put in the record, and 
then I will proceed to provide a short verbal summary of the sum- 
mary. 

Chairman Levin. All the statements will be made part of the 
record. Thank you. 

Dr. Kaminski. Thank you, sir. 

Recent years have seen a serious erosion in our ability to field 
new weapon systems quickly in response to changing threats, as 
well as a large increase in the cost of these weapon systems. Our 
programs today for developing weapon systems take two to three 
times longer than they did 30 years ago. I note that time is money 
in this process, and time also leaves room for disruptions, uncer- 
tainty, and changes in commercial technology. In a 15- or 20-year 
period, we’re seeing commercial technology turnover three, four, or 
five times. When a weapon system takes 15 to 20 years to develop, 
the technology that you start with isn’t going to be supported when 
it’s fielded. So, we have to vigorously attack this time issue. 

Our committee also noted the importance of systems engineering 
in reducing this acquisition time, when combined with development 
planning. We further underscored the importance of an early sys- 
tems engineering effort, in that, the decisions made prior to and 
the key Milestone A and B decisions impact somewhere between 75 
percent and 85 percent of the total life-cycle cost. So, the time to 
address those issues is up front, before those decisions are made. 

Our committee also noted that many of the conclusions that we 
reached have been reached in several previous studies. So, the 
issue isn’t disagreement on what the recommendations are, the 
issue is implementing those recommendations. So, once again, we 
thank you for your leadership in creating a forum for that kind of 
implementation. 

Let me address now the issues that you asked me to address. 

First of all, just one overall comment on systems engineering. I 
agree with Secretary Gates, who, when asked about acquisition, 
said, “There is no one silver bullet that is going to correct all the 
problems.” But, I do believe that good systems engineering, coupled 
with effective development planning early on, are two of the most 
important contributors to successful acquisition. 

Our report provided some formal definitions of “systems engi- 
neering,” but they tend to be arcane, so I thought I might start 
with a couple of examples. I’ll briefly describe some examples of 
good systems engineering in the work we’ve done, and also where 
have we seen poor systems engineering. 
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One of the really good examples is the Apollo program. That pro- 
gram, from a dead start, put man on the moon in about 8 years. 
When that program was started, we didn’t have mature technology. 
What we did was good upfront systems engineering and develop- 
ment planning, so we could proceed in a sequential way, step by 
step, with each new step building on the previous step. In building 
hardware, we were also building the experience of our acquisition 
workforce and our industry, so we could, step by step, increase our 
capabilities, eventually going to the moon. 

Another really good example is the Air Force Intercontinental 
Ballistic Missile (ICBM) programs that were done in the 1970s and 
1980s. What we saw there is that we would never start a full-scale 
development contract for a new ICBM until we had done the up- 
front systems engineering and development planning. The develop- 
ment planning produced an inertial guidance system for the ICBM, 
as well as critical propulsion components, and a reentry vehicle. 
That not only reduced the risk of the hardware development and 
integration in the future, it also gave domain experience to our key 
people in government and industry, so that when we threw the 
switch and started full-scale development, we could typically expect 
a first flight in 3 to 4 years, as a result of that experience base. 
That’s what we need to restore. 

You asked: “What were systemic contributors to acquisition prob- 
lems?” I listed five. 

The first of these is the lack of this early and continuing systems 
engineering, coupled to a development planning program early on, 
right upfront. 

The second key impediment is the lack of alignment of responsi- 
bility, authority, and accountability of the program manager. A pro- 
gram manager needs to be able to exercise his or her judgment. 
Much of the program manager’s authority has been taken away by 
one-size-fits-all approaches to acquisition and by the oversight proc- 
ess, which has some onerous elements that are nonvalue-added. 

A third major impediment is the lack of stability in program 
funding. Many contribute to that. 

A fourth is the lack of early attention to test and evaluation, 
with insufficient planning and investment in the tools, such as 
modeling and simulation, test equipment, facilities, and personnel, 
to provide us with the timely and meaningful results needed by 
program management and for continuous systems engineering to 
refine our performance objectives and development plans. 

Finally, the root fundamental issue here is this excessive time to 
acquire that I had spoken about. Time is money. As this time in- 
creases from a few years in the past to 15 years today, it under- 
mines our entire process, causing the key participants to lose what 
I call the “recipe” for how we move forward and also to lose a sense 
of accountability. When we see new capabilities that are developed 
and fielded in 5 years, the engineers, the managers, the testers, the 
cost analysts are all able to benefit and apply their experience from 
previous programs, and they can also be held accountable, since 
they can be in place managing the programs deliverables during 
one assignment. That all changes when we move to 15-year acqui- 
sitions and we have five rollovers of management, engineers, and 
cost analysts, and five rollovers of the technology in the process. 



27 


So, attacking acquisition time is fundamental. I would say, a tes- 
tament to our failure today is the fact that we have to discard our 
current acquisition approaches to deal with our urgent needs and 
field systems, such as the Mine Resistant Ambush Protected vehi- 
cle and counter-improvised explosive devices (lEDs) by forming and 
using rapid-reaction organizations, because our existing ones don’t 
work. They can’t respond to the cycle time that we need. 

So, what do we do about this? Again, I’ve listed in my statement 
five steps. 

The first is to ensure that we not only restore, but enhance, this 
early and continuing systems engineering work, coupled with de- 
velopment planning. This means restoring funding upfront in the 
programs, and using independent estimates to ensure we have 
enough funding upfront. It also means attracting best and bright- 
est to the critical systems engineering work, and providing a path 
to career advancement, career tracking, and leadership for the key 
people that we need to rebuild in cadre. 

Second issue is the alignment of the responsibility, authority, 
and accountability of the program manager. I’ve listed several steps 
in my statement about what’s needed to be done to do that. 

Third issue is improving funding stability. We pay a great deal 
for the instability we cause by making funding adjustments to pro- 
gram. My experience shows that every time we make a cut in a 
program, for financial or other reasons unrelated to performance, 
we end up eventually putting in three times what we cut to restore 
the program later and get it back to a base. 

Fourth item is giving early and serious attention to the test and 
evaluation issues that I noted earlier, so they can be part of a rapid 
process. When we wait for test and evaluation results because we 
haven’t done a good job planning and preparing, what we have is 
hundreds of people sitting on their hands, waiting for results, and 
we’re paying all those people while they wait for results. 

Finally, the last item is fundamentally attacking this problem of 
long development times by the combination of the previous four 
items. 

I believe action on these five issues will have a significant and 
demonstrable impact on our serious acquisition problems. I believe 
that we need to move now with the same urgency and priority that 
we expect in combat operations to permit the timely and effective 
development and fielding of new capabilities and services with 
what I expect will be more limited future defense dollars. 

Thank you, Mr. Chairman. 

[The prepared statement of Dr. Kaminski follows:] 

Prepared Statement by Dr. Paul G. Kaminski 

Chairman Levin, and Ranking Member McCain: 

Thank you for your leadership on the Department of Defense (DOD) acquisition, 
and for the invitation to testify on these important acquisition issues. Since you 
have asked me to testify in my role as Chair, Committee on Pre-Milestone A Sys- 
tems Engineering, Air Force Studies Board, National Research Council, I will begin 
by providing a summary of our report, which was approved by the Governing Board 
of the National Research Council and published in 2008. The report is available to 
the public at http://www.nap.edu. After the report summary, I will provide my per- 
sonal views on systems engineering and respond to the key issues you asked that 
I address. 
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SUMMARY 

Recent years have seen serious erosion in the ability of U.S. forces to field new 
weapons systems quickly in response to changing threats, as well as a large increase 
in the cost of these weapons systems. Today the military’s programs for developing 
weapons systems take two to three times longer to move from program initiation 
to system deployment than they did 30 years ago. This slowdown has occurred dur- 
ing a period in which threats have been changing more rapidly than ever and when 
technology advances and accumulated experience should have been accelerating 
rather than slowing the development process. 

Many causes for this trend have been suggested, including the increased com- 
plexity of the tasks and the systems involved from both technological and human/ 
organizational perspectives; funding instability; loss of “mission urgency” after the 
end of the Cold War; bureaucracy, which increases cost and schedule but not value; 
and the need to satisfy the demands of an increasingly diverse user community. The 
difficulty of focusing on a specific, homogeneous, post-Cold War threat made prob- 
lems even worse. Yet although the suggested causal factors have merit, a common 
view is that better systems engineering (SE) and development planning could help 
shorten the time required for development, making it more like what it was 30 
years ago. 

Simply stated, SE is the translation of a user’s needs into a definition of a system 
and its architecture through an iterative process that results in an effective system 
design. SE applies over the entire program life cycle, from concept development to 
final disposal. 

The Committee on Pre-Milestone A Systems Engineering was tasked by the U.S. 
Air Force to examine the role that SE can play during the defense acquisition life 
cycle in addressing the root causes of program failure, especially during the pre- 
Milestone A and early phases of a program. Currently, few formal SE processes are 
applied to Air Force development programs before the Milestone A review. ^ 

The committee devoted considerable time and space in its report to tr 3 dng to de- 
fine a minimum set of systems engineering processes. The most important of these 
processes are summarized in the checklist in Box S-1 below. A few of the things 
that need to be taken care of before Milestone A and just after it are the following: 
the consideration of alternative concepts (solutions) up front; the setting of clear, 
comprehensive key performance parameters (KPPs) and system requirements; and 
early attention to interfaces and interface complexity, to the concept of operations, 
and to the system verification approach. It is these early-stage processes that are 
covered in this report. The importance of stable requirements and funding between 
Milestone B and the achievement of initial operational capability (IOC) is stressed, 
as are processes including good configuration management and change control. The 
committee further stresses in the report what it regards as six of the most impor- 
tant process areas in its discussion of six “seeds of failure”. 


^This is a result of the elimination in the 1990s of the development planning function that 
had existed in the Air Force Systems Command. 
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Figure S-1 DOD life-cycle acquisition process. Points A, B, and C at the top of 
the figure represent Milestones A, B, and C. LCC, life-cycle cost. Source: Richard 
Andrews, 2003, An Overview of Acquisition Logistics. Fort Belvoir, VA; Defense Ac- 
quisition University. Available at http://www.afcea.org/events/pastevents/documents/ 
Track4Session4AMCEmphasisonCustomerFocusedITInitiatives.ppt#364, 12, Slide 12. 
Last accessed on November 20, 2007. 

SYSTEMS ENGINEERING AND THE DOD ACQUISITION LIFE CYCLE 

The use of formal systems engineering practices throughout the life cycle of an 
acquisition program is critical to fielding the required system on time and within 
budget. Across the top of Figure S-1 are the points at which important management 
decisions are made: Milestones A, B, and C. Concept development and refinement 
occur before Milestone A, and further technology development, to reduce system de- 
sign and development (SDD) risk, occurs before Milestone B. Only after Milestone 
B does a program become an enterprise with dedicated funding. Importantly, Figure 
S-1 shows that about three-quarters of total system life-cycle costs are influenced 
by decisions made before the end of the concept refinement phase at Milestone A, 
while about three-quarters of life-cycle funds are not actually spent until after Mile- 
stone C. This means that although high-quality SE is necessary during the entire 
acquisition cycle, the application of SE to decisions made in the pre-Milestone A pe- 
riod is critical to avoiding (or at least minimizing) cost and schedule overruns later 
in a program. Much of the value of early, high-quality SE will be manifested as suc- 
cess in fulfilling Milestone B requirements. 

MAIN FINDINGS AND RECOMMENDATIONS 

The committee’s main findings and recommendations are given below. 

Finding 

Attention to a few critical systems engineering processes and functions particu- 
larly during preparation for Milestones A and B is essential to ensuring that Air 
Force acquisition programs deliver products on time and on budget. 

Today’s weapons systems provide unprecedented capabilities but also involve com- 
plex interfaces with external command, control, and communications systems and 
rely on a greater volume of software than ever before. Early decisions on the weap- 
ons system requirements and capabilities have a disproportionately large impact on 
program cost and schedule. The committee also recognizes that a lack of flexibility 
(a result of overly rigid processes or a lack of trust among program participants or 
stakeholders) can limit the ability of a program manager to change early decisions 
that warrant changing. 

The committee found many gaps and inconsistencies in the way that the Air Force 
manages pre-Milestone A activities. The committee heard from presenters of some 
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cases for which required documents were completed pro forma and filed away, never 
to be seen again, or for which required steps were skipped completely. The current 
practice of initiating programs at Milestone B denies the acquisition review author- 
ity the earlier opportunity (at Milestone A) to make judgments about the maturity 
of the technologies on which the program is based and to decide whether tech- 
nologies need to be further developed prior to making a Milestone B commitment 
to system development and demonstration. 

Recommendation 

The Air Force leadership should require that Milestones A and B be treated as 
critical milestones in every acquisition program and that a checklist such as the 
“Pre-Milestone A/B Checklist” suggested by the committee (see Box S-1 in this 
Summary) be used to judge successful completion. 

A rigorous, standard checklist of systems engineering issues should be addressed 
by each program through both the pre-Milestone A and pre-Milestone B phases. The 
committee’s recommended 20-item checklist is shown in Box S-1. 


Box S-1 

Pre-Milestone A/B Checklist 
Concept Development 

1 . Have at least two alternative concepts to meet the need been evaluated? 

The purpose of alternatives is to stimulate thinking to find the simplest, fastest, and 
cheapest solution. 

2. Can an initial capability be achieved within the time that the key program leaders are 
expected to remain engaged in their current jobs (normally less than 5 years or so after 
Milestone B)? If this is not possible for a complex major development program, can 
critical subsystems, or at least a key subset of them, be demonstrated within that time 
frame? 
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Achieving capabilities or demonstrating critical subsystems while key programs leaders 
remain engaged is important to get the capability into service quickly and cost- 
effectively and to begin the process of incremental improvements based on operational 
experience. 

3. Will risky new technology have been matured before Milestone B? If not, is there an 
adequate risk mitigation plan? 

The development of risky new technology in parallel with a major development program 
can be costly in terms of both time and money. 

4. Have external interface complexities (including dependencies on other programs) been 
identified and minimized? Is there a plan to mitigate their risks? 

Complex, ill-defined, external requirements and interfaces can be a major source of 
requirements instability during the development phase. This can be of particular 
importance when a system must operate in a system-of-systems environment. 

Key Performance Parameters and CONOPS 

5. At Milestone A, have the KPPs been identified in clear, comprehensive, concise terms 
that are understandable to the users of the system? 

It is important that KPPs be expressed in terms understandable to all of the 
stakeholders. Failure to define the system ’s KPPs simply and clearly at Milestone A is a 
first step to requirements instability and overruns later. 

6. At Milestone B, are the major system-level requirements (including all KPPs) defined 
sufficiently to provide a stable basis for the development through IOC? 

Beginning development without a complete list of stable requirements is one of the key 
"seeds of failure" described in Chapter 4 in this report. It is important to complete 
requirements trade-offs prior to the development phase. 

7. Has a CONOPS been developed showing that the system can be operated to handle the 
expected throughput and meet response time requirements? 

It can be costly to discover too late that the system as designed cannot be operated to 
meet its requirements. 

Cost and Schedule Scoping 

8. Are the major known cost and schedule drivers and risks explicitly identified, and is 
there a plan to track and reduce uncertainty? 

Identifying the major cost and schedule risk areas, with particular attention to this 
checklist and the six seeds of failure — inexperienced leadership, external interface 
complexity, system complexity, incomplete requirements at Milestone B, immature 
technology, and high reliance on new software — can help focus management on these 
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issues early. 

9. Has the cost confidence level been accepted by the stakeholders for the program? 

Il is important that stakeholders understand the degree of risk so that the stakeholders 
will not disrupt the program as the inevitable development program surprises unfold 
later on. It will generally not be possible by Milestone A or Milestone B to identify all 
the risk areas that might surface later in a development program, but a frank, early 
disclosure of known potentials for risk can help sustain stakeholder support later on. 

Performance Assessment 

10. Is there a sufficient collection of models and an appropriate simulation environment to 
validate the selected concept and the CONOPS against the KPPs? 

In large, complex programs, the development of models early on can be very important 
to later management of requirements changes and performance verification. 

1 1. At Milestone B, do the requirements take into account likely future mission growth over 
the program life cycle? 

The committee advocates freezing new requirements and new technology insertion after 
Milestone B but also notes that making provisions in the initial requirements to facilitate 
later upgrades could have great long-term value. 

Architecture Development 

12. Has the system been partitioned to define segments that can be independently developed 
and tested to the greatest degree possible? 

Effective partitioning of a complex system can greatly reduce its development cost. 

13. By Milestone A, is there a plan to have information exchange protocols established for 
the whole system and its segments by Milestone B? 

Such a plan developed early on can greatly reduce interface problems later in the 
development phase when they would be more difficult and costly to fix. 

14. At Milestone B, has the government structured the program plan to ensure that the 
contractor addresses the decomposition of requirements to hardware and software 
elements sufficiently early in the development program? 

The histories of programs with cost and schedule overruns are replete with examples of 
large software developments that had to be redone because requirements from the 
hardware side were assigned or determined late. 

Risk Assessment 

15. Have the key risk drivers (not only the technology drivers) been identified? 
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Identifying and managing risk early can pay large dividends: it is important to focus on 
the six "seeds of failure " (see item 8 above). 

Program Implementation Strategy 

16. Does the government have access over the life of the program to the talent required to 
manage the program? Does it have a strategy over the life of the program for using the 
best people available in the government, the FFRDCs, and the professional service 
industry? 

Seasoned management is critical; the government’s job is to find the best! 

17. At Milestone A, is there a plan defining how the pre-Milestone B activity will be done, 
and by whom? 

Identifying the program and system managers early, identifying the FFRDC or SETA 
support needed, thinking through the use of competitive system concept contracts — ail 
can have a decisive impact on the government's ability to select the best concept, to 
define by Milestone B system requirements that can remain stable through IOC, and to 
select the best development contractors. 

18. Is there a top-level plan for how the total system will be integrated and tested? 

A well-thought-out strategy for verifying system performance, including optimum 
phasing of verification tests throughout the assembly process, and well-thought-out use 
of analytical models and external simulators can have a large positive impact on 
ultimate cost, schedule, and performance. 

19. At Milestone B, have sufficiently talented and experienced program and systems 
engineering managers been identified? Have they been empowered to tailor processes 
and to enforce requirements stability from Milestone B through IOC? 

Seasoned leaders in these areas are critical to maintaining focus and discipline through 
IOC. 

20. Has the government attempted to align the duration of the program manager’s 
assignment with key deliverables and milestones in the program? 

A combination of assignment extension and time-certain milestones will help align 
incentives. 

NOTE: KPP, key performance parameter; CONORS, concept of operations; IOC, initial 
operational capability; FFRDC, federally funded research and development center; SETA, 
systems engineering and technical assistance. 


While the committee considers that each item on the checklist is important, it 
calls attention to several items that warrant further discussion. Item 2 recognizes 
that the world changes too fast to be friendly to long development cycles. The com- 
mittee believes that the Air Force should strive to structure major development pro- 
grams so that initial deployment is achieved within, say, 3 to 7 years. Thirty years 
ago, this was a typical accomplishment — for example, nearly 40 years ago, the Apol- 
lo program put the first man on the Moon in fewer than 8 years. 

The development time issue is addressable by applying systems engineering to 
Items 3, 4, and 13 through 15 before Milestones A and B. The definition of clear 
KPPs by Milestone A and clear requirements by Milestone B that can remain stable 
through IOC can be essential to an efficient development phase. It is also important 
that critical technologies be sufficiently mature prior to starting SDD. The com- 
mittee observed that although today’s systems are not necessarily more complex in- 
ternally than those of 30 years ago, their “external complexity” often is greater, be- 
cause today’s systems are more likely to try to meet many diverse and sometimes 
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contradictory requirements from multiple users. This kind of complexity can often 
lead to requirements being changed between Milestone B and IOC, and it can lead 
to relying on immature technology. 

Item 19 of the checklist stresses the importance of placing experienced, domain- 
knowledgeable managers in key program positions. The committee has observed 
that many of the truly extraordinary development programs of the past, such as 
Apollo, the Manhattan Project, the early imaging satellite programs, the U-2, the 
fleet ballistic missile system, and nuclear submarines, were managed by relatively 
small (and often immature) agencies with few established processes and controls. In 
that environment, dedicated managers driven by urgent missions accomplished feats 
that often seem incredible today. 

The committee believes that the accumulation of processes and controls over the 
years — well meant, of course — has stifled domain-based judgment that is necessary 
for timely success. Formal SE processes should be tailored to the application. But 
they cannot replace domain expertise. In connection with item 19, the committee 
recommends that the Air Force place great emphasis on putting seasoned, domain- 
knowledgeable personnel in key positions — particularly the program manager, the 
chief system engineer, and the person in charge of “requirements” — and then em- 
power them to tailor standardized processes and procedures as they feel is nec- 
essary. 

One key pre-Milestone A task is the analysis of alternatives (AoA), which entails 
evaluating alternative concepts and comparing them in terms of capabilities, costs, 
risks, and so on. Checklist items 1 through 4, 12, and 13 should be completed before 
the AoA, while items 5 through 11 and 14 through 20 may be addressed after the 
AoA. 

Finding 

The creation of a robust systems engineering process is critically dependent on 
having experienced systems engineers with adequate knowledge of the domain rel- 
evant to a contemplated program. 

While the systems engineering process has broad use, effective application de- 
pends on having domain experts who are aware of what has gone wrong (and right) 
in the past, recognize the potential to repeat the successes under new cir- 
cumstances, and avoid repeating the errors. 

Ideally, a person or persons with domain knowledge would have had experience 
working on exactly the same problem, or at least a problem related to the one at 
hand. If that is not so (and it might not be if the problem has never been addressed 
before, as was the case for Apollo and nuclear submarines), the term could be taken 
to refer to academic training in the relevant field of engineering or science. It could 
also refer to the practice of critical thinking and problem solving that comes with 
learning to be a systems engineer and then building on that foundation to gain the 
experiential knowledge and understanding of engineering in the context of an entire 
system. Systems engineering is enabled by tools that have been developed to assist 
in the management of systems engineering (not to be confused with the practice of 
systems engineering). 

Both industry and Air Force presenters told the committee that there are not 
enough domain-knowledgeable and experienced systems engineers to support all of 
the programs that need them. 

Recommendation 

The Air Force should assess its needs for officers and civilians in the systems en- 
gineering field and evaluate whether either its internal training programs, which 
include assignments on Air Force programs that provide mentoring by experienced 
people and hands-on experience in the application of systems engineering principles, 
or external organizations are able to produce the required quality and quantity of 
systems engineers and systems engineering skills. Based on this assessment, the Air 
Force first should determine how and where students should be trained, in what 
numbers, and at what cost, and then implement a program that meets its needs. 

The Air Force needs to attract, develop, reward, and retain systems engineers 
across the full spectrum of relevant domains, engage them in the early (pre-Mile- 
stone A) phase of new programs (or modification programs), and sustain their par- 
ticipation throughout the life of the programs. One important step in this process 
would be to create an Air Force occupational code for systems engineering so that 
engineers’ experience and education can be tracked and managed more effectively. 
The Air Force should support an internal systems engineering career track that re- 
wards the mentoring of junior systems engineering personnel, provides engineers 
with broad systems engineering experience, provides appropriate financial com- 
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pensation to senior systems engineers, and enables an engineering career path into 
program management and operations. 

Finding 

The Government, federally Funded Research and Development Centers (FFRDCs), 
and industry all have important roles to play throughout the acquisition life cycle 
of modern weapons systems. 

Since the need for a new or upgraded weapons system is most often first recog- 
nized by the military user, it is appropriate for the military to codify its require- 
ments and, with support from FFRDC and independent systems engineering and 
technical assistance contractors, to explore materiel and nonmateriel solutions (such 
as doctrinal, organizational, or procedural changes) as well as to assess the potential 
for new technology to provide enhanced capabilities. While it is appropriate and 
usually desirable to engage development contractors in the pre-Milestone B process 
using competitive study contracts, the source selection for system development and 
demonstration should not be made until after the work associated with Milestones 
A and B is complete. 

Recommendation 

Decisions made prior to Milestone A should be supported by a rigorous systems 
analysis and systems engineering process involving teams of users, acquirers, and 
industry representatives. 

Working together, government and industry can develop and explore solutions 
using systems engineering methodology to arrive at an optimal systems solution. 

Finding 

The Air Force used to have a development planning organization that applied pre- 
Milestone A systems engineering processes to a number of successful programs, but 
that organization was allowed to lapse. 

The role of the Air Force development planning organization, which was within 
the Air Force Systems Command, was to provide standard evaluation tools and per- 
form pre-Milestone A systems engineering functions across acquisition programs. 
The early 1990s saw an erosion of this front-end planning organization along with 
its funding as the Air Force Systems Command (now the Air Force Materiel Com- 
mand) began to play a decreasing role in program execution. In the opinion of sev- 
eral speakers who met with the committee, one main reason for the erosion of fund- 
ing was a lack of congressional support for the planning function. 

Recommendation 

A development planning function should be established in the military depart- 
ments to coordinate the concept development and refinement phase of all acquisition 
programs to ensure that the capabilities required by the country as a whole are con- 
sidered and that unifying strategies such as network-centric operations and inter- 
operability are addressed. 

The Air Force and the other military services should establish a development 
planning organization like that which existed in the early 1990s. 

The roles and functions of the various organizations involved in acquiring major 
weapons systems need to be clearly defined. The responsibility for executing systems 
engineering and program management in the pre-Milestone A and B phases should 
be vested in the military departments that do the actual development planning 
functions. This should not be the responsibility of the Office of the Secretary of De- 
fense (OSD) or of the Joint Staff. Instead, those offices need to enable the creation 
and functioning of military department development planning organizations with 
policy measures and, where appropriate, resources. The Joint Staff, under the aus- 
pices of the Joint Requirements Oversight Council, may help to define the require- 
ments for major programs in the course of the development planning process, but 
it should not run the process itself. 

The existence of “joint” programs or a program such as Missile Defense, which 
has several related systems being developed by different military services, requires 
clear guidance from both OSD and the Joint Staff about who is in charge. These 
programs need to be harmonized and integrated by the responsible integrating 
agency. However, development planning activities should still take place in the mili- 
tary departments where the expertise resides. Consequently, the development plan- 
ning should be managed by that agency. 

While this committee cannot predict how Congress will view the revival of a good 
planning process to support pre-Milestone A program efforts, it is still important for 
the Air Force and DOD to make the case for the critical importance of this process 
before Congress and others. A development planning process is important not to 
start new programs, but rather to ensure that any new program (or a new start of 
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any kind) is initiated with the foundation needed for success. Funding for this plan- 
ning function needs to be determined by the military services, including both the 
acquisition communities and those (the warfighters) who generate the operational 
requirements. 


CONCLUDING THOUGHTS 

Many of the conclusions reached and recommendations made by the committee 
are similar to those of previous reviews. Most of the past recommendations were 
never implemented, so one of this committee’s most critical thoughts relates to the 
importance of implementation. A sampling of key findings and recommendations 
from previous studies follows: 

• Government Accountability Office (GAO) 

• Separate technology development from systems acquisition. Commit to a 
program only if the technology is sufficiently mature. Set the minimum 
Technology Readiness Level (TRL). 

• Stabilize the requirements early. 

• Employ systems engineering techniques before committing to product de- 
velopment. 

• Employ evolutionary approaches that pursue incremental increases in ca- 
pability. 

• Address shortfalls in science, engineering, and program management 
staff. 

• National Defense Industrial Association (NDIA) * 

• Increase SE awareness and recognize SE authority in the program formu- 
lation and decision process. 

• Incentivize career SE positions within the government. 

• Defense Science Board (DSB)® 

• Overhaul the requirements process. 

• Stabilize acquisition tours. 

• Establish a robust SE capability. 

• Defense Acquisition Performance Assessment (DAP A) ® 

• Strategic technology exploitation is a key U.S. advantage. Opportunities 
need to be identified early. 

• The U.S. economic and security environments have changed — for exam- 
ple, there are fewer prime contractors, smaller production runs, reduced 
plant capacity, fewer programs, and unpredictable threats. 

• The acquisition system must deal with instability of external funding. 

• The DOD management model is based on a lack of trust. Quantity of re- 
view has replaced quality. There is no clear line of responsibility, authority, 
or accountability. 

• Oversight is preferred to accountability. 

• Oversight is complex, not process- or program-focused (as it should be). 

• The complexity of the acquisition process increases costs and draws out 
the schedule. 

• Incremental improvement applied solely to the “little a” acquisition proc- 
ess requires all processes to be stable — but they are not. 

The committee notes that successful implementation of these recommendations re- 
quires the “zipper concept” — making connections at all levels, from the senior lead- 


2 GAO, 2003, Defense Acquisitions: Improvements Needed in Space Systems Acquisition Man- 
agement Policy, September. Available at http://www.gao.gov/new.items/d031073.pdf. Last 
accessed April 2, 2007. 

3 GAO, 2005, Space Acquisitions: Stronger Development Practices and Investment Planning 
Needed to Address Continuing Problems, July. Available at http://www.gao.gov/new.items/ 
d05891t.pdf. Last accessed April 2, 2007. 

^NDIA Systems Engineering Division, 2003, Task Report: Top Five Systems Engineering 
Issues in Defense Industry, January, Arlington, VA: NDIA. 

® DSB/Air Force Scientific Advisory Board (AFSAB) Joint Task Force, 2003, Acquisition of Na- 
tional Security Space Programs, May, Washington, DC. 

® Ronald Kadish, Gerald Abbott, Frank Cappuccio, Richard Hawley, Paul Kern, and Donald 
Kozlowski. 2006. Defense Acquisition Performance Assessment. Available at http:// 
www.acq.osd.mil/dapaproject/documents/DAPA-Report-web/DAPA-Report-web-feb21.pdf. Last 
accessed on April 2, 2007. 

"^The Acquisition — “Big A” — system is often believed to be a simple construct that efficiently 
integrates three independent processes: requirements, budgeting, and acquisition. “Little a” on 
the other hand, refers to the acquisition process that focuses on “how to buy” in an effort to 
balance cost, schedule, and performance; it does not include requirements and budgeting. 
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ership of the Air Force and DOD down to the working levels within key program 
management offices and supervisory staffs. 

COMMITTEE ON PRE-MILESTONE A SYSTEMS ENGINEERING: A RETROSPECTIVE REVIEW 
AND BENEFITS FOR FUTURE AIR FORCE SYSTEMS ACQUISITION 

Paul Kaminski, Chair, Technovation, Inc. 

Lester Lyles, Vice Chair, U.S. Air Force, (Ret.) 

Dev Banerjee, The Boeing Company 

Thomas Blakely, Lockheed Martin Aeronautics Company 

Natalie Crawford, RAND Corporation 

Stephen Cross, Georgia Tech Research Institute 

Gilbert Decker, Independent Consultant 

Llewelln Dougherty, Raytheon 

John Farr, Stevens Institute of Technology 

James Frey, Frey Associates 

Robert Fuhrman, Lockheed Martin (Ret.) 

David Gorney, The Aerospace Corporation 

John Griffin, Griffin Consulting 

Wesley Harris, Massachusetts Institute of Technology 
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Mark Wilson, Mark Wilson Consulting 

Staff 

Michael A. Clarke, Board Director 
James C. Garcia, Study Director 
Greg Eyeing, Senior Program Officer 
Carter W. Ford, Associate Program Officer 
William E. Campbell, Senior Program Associate 
Lanita Jones, Program Associate 

Having summarized the findings and recommendations of the Committee on Pre- 
Milestone A Systems Engineering, let me now add my personal views on systems 
engineering, and the two additional questions that you asked me to address: 1) the 
systemic issues that have contributed to cost, schedule, and performance problems 
in the acquisition of major weapon systems; and 2) the steps that Congress and the 
Department need to take to improve performance of the Department’s acquisition 
programs. 


SYSTEMS engineering 

I agree with Secretary Gates who said that there is no one silver bullet that will 
correct all of the DOD acquisition problems. But I believe that good systems engi- 
neering coupled with effective development planning are the two most important 
contributors to successful acquisition. Our report provided formal definitions of sys- 
tems engineering and development planning that are somewhat arcane. So rather 
than provide further definition, I find it easier to illustrate by choosing examples 
of good and bad systems engineering and development planning. Examples of good 
work include the Apollo Program and the U.S. Air Force intercontinental ballistic 
missile (ICBM) programs (e.g., Minuteman, MX) in the 1970s-1980s. Apollo suc- 
ceeded in putting men on the moon in about 8 years. At the start of the program, 
almost all of the key technologies were immature. But good systems engineering and 
development planning were applied to develop a systematic approach, reducing risk 
by taking a series of limited steps, and applying the learning and domain experience 
gained from each step to the subsequent step. The U.S. Air Force ICBM programs 
used a similar approach, beginning with conceptual studies and technology develop- 
ment, and holding initiation of full scale development (FSD) contracts until key 
guidance system, re-entry system and propulsion technologies had been dem- 
onstrated. As a result, the time from initiation of FSD until first flight was typically 
Z-A years. An example of poor systems engineering is the SBIRS program, in which 
a lack of domain experience and analysis led to a failure to anticipate the possibility 
of severe radio frequency interference between two key payloads — discovering this 
problem years after program initiation. Inadequate systems engineering and devel- 
opment are therefore the first of five items listed below as systemic contributors to 
acquisition problems. 
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SYSTEMIC CONTRIBUTORS TO ACQUISITION PROBLEMS 

1. The lack of early and continuing systems engineering and the absence 
of a closely-coupled development planning program are a fundamental con- 
tributor, as identified in our report. The root causes include: (a) lack of suf- 
ficient personnel (in both government and industry) with adequate edu- 
cation, training and domain experience (this includes personnel in require- 
ments development as well as in acquisition); and (b) lack of sufficient front 
end investment necessary to understand the key tradeoffs in cost/schedule/ 
performance, and to identify and address the key risks in a systematic 
manner. 

2. Lack of alignment of responsibility, authority, and accountability of the 
program manager. In many cases the program manager’s authority is dif- 
fused by many levels of oversight in both the Department and in Congress, 
and the financial and performance constraints imposed do not allow suffi- 
cient freedom of action to apply informed judgment in a timely manner. 
Flexibility is further limited by application of a “one-size-fits-all” approach 
imposed by the DOD 5000 system, and the oversight practiced by the DOD 
and Congress. A program manager needs the freedom to tailor the acquisi- 
tion approach to the problem, to ensure that the program response time will 
fall within the response time of the threat, and to apply a variety of tools 
and techniques (such as the use of prototypes, competitive prototypes, mod- 
eling and simulation, critical subsystem and component demostration). For 
this to work, we need program managers with the education, training, and 
domain experience needed to enable timely responses and excellent judg- 
ment relevant to the domain. 

3. Lack of stability in program funding. 

4. Lack of early attention to test and evaluation, with insufficient plan- 
ning and investment in the tools (e.g., modeling and simulation, test equip- 
ment, facilities, and personnel) to provide the timely and meaningful re- 
sults needed by program management and systems engineering to contin- 
ually refine performance objectives and development plans. 

5. Excessive (and growing) time from program initiation to fielding. As 
this time increases from a few years to 15 years or more, it undermines the 
entire acquisition process by causing key participants to “lose the recipe”, 
and lose a sense of accountability as well as a sense of being able to make 
a difference. When new capabilities are developed and fielded in 5 years, 
engineers, managers, testers, cost analysts, etc. are able to benefit from and 
apply the experience gained from a previous program or program phase. 
They can also see the results of their decisions and be held accountable. We 
can also meaningfully employ past performance of the contractor as a factor 
in the award of future programs — an important factor in incentivizing con- 
tractor performance. This all changes dramatically when the time extends 
to 15 years, and we have five roll-overs of management, engineers, cost ana- 
lysts, and commercial technology during this time period. This long and 
growing time period is a result of the inflexibility inherent in our entire 
system of requirements development, budgeting and acquisition, and it cre- 
ates a vicious cycle in which it further exacerbates the contributors above, 
and they in turn further increase the time and cost growth. We see the re- 
sult when we must discard our current acquisition system in order to deal 
with urgent needs and field systems such as MRAP and jammers to counter 
lED’s by forming and using rapid reaction organizations. This cycle must 
be broken by attacking the root causes. 

STEPS THAT CONGRESS AND THE DEPARTMENT NEED TO TAKE 

1. The first step is to insure that we not only restore, but enhance early 
and continuing systems engineering coupled with effective development 
planning. This will require commitment of more significant investment dol- 
lars earlier in our acquisition programs, and a commitment to build a cadre 
of systems engineers and development planners with the education, train- 
ing and domain experience needed to be effective. Attracting “best and 
brightest” to this work — and keeping them — will require a personnel system 
that will identify and track these important human resources and establish 
a career path to allow those who are successful to advance to senior pro- 
gram management and leadership positions. Their domain experience will 
be enhanced by managing the building of critical subsystems during the de- 
velopment planning program, reducing risk and building skills and experi- 
ence at the same time. Congress and the Department can assist by pro- 
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viding incentives for attracting and keeping key personnel (not only finan- 
cial incentives, but educational, training, recognition, and most important — 
the opportunity to take on challenging developments and see that they can 
make a difference). We will need metrics to assess how well we are doing 
in building and appl^ng this cadre, and we must recognize that this will 
not be accomplished in 4, or even 8 years. But we must begin in earnest 
and begin now. Finally, we need a means to insure that we have adequate 
funding upfront for new programs; one approach would require a report at 
program initiation from an independent cost analyst working with system 
engineers and development planners who have developed their skills on 
previous programs. 

2. Alignment of the responsibility, authority, and accountability of the 
program manager requires that a degree of trust be established between 
the program manager and those responsible for our oversight mechanisms. 
We must be prepared to delegate authority to the program manager, and 
provide him or her with some flexibility to manage — to adjust levels and al- 
location of funding, to adjust the allocation of performance parameters, to 
adjust schedule, and to tailor the acquisition approach to be responsive to 
the need. Clearly, there must be bounds established beyond which the pro- 
gram manager must seek approval from oversight authorities. But I believe 
these bounds are too narrow and inflexible today. One size does not fit all 
programs. Congress and the Department should be willing to consider and 
tailor many of the restrictions which unnecessarily limit and delay program 
managers today. I have seen many of our successful classified special pro- 
grams benefit from greater management flexibility than that afforded to 
their in conventional program counterparts. The good managers of these 
special programs have used that flexibility to the benefit of the program 
and the Department by operating with transparency and maintaining trust. 
I realize that it seems counterintuitive to recommend greater flexibility and 
trust in an environment rife with acquisition problems, but I believe we 
need to break the current cycle. One way to begin is with a limited number 
of pilot programs, with first priority to those programs addressing urgent 
needs, and assignment of our most experienced program managers to meet 
those urgent needs. Since these programs will be moving with dispatch, 
they offer the best opportunity to produce early indications of whether this 
is a sound approach which should be extended to other programs. 

3. Improving funding stability will require that the Department and Con- 
gress be willing to give up some of their flexibility in making annual (or 
more frequent) adjustments in funding. Doing so will require tradeoffs of 
the costs and benefits, and I believe it is time to make explicit consideration 
of these tradeoffs. I have seen the projected benefits of stable funding by 
looking at theoretical Monte-Carlo simulations (which show efficiency im- 
provements of perhaps 8-10 percent as a result of holding a small capital 
Reserve of less than 10 percent). We can also see the benefits of multiyear 
procurements saving similar or greater amounts. I have also seen many ex- 
amples in which funding cuts of x dollars today result in later additions of 
3x dollars to catch up. 

4. Giving early and serious attention to test and evaluation will require 
strengthening our test and evaluation organizations and personnel. Test 
and evaluation is often an afterthought, and contracts are often written 
without any mention of how we will test the product. We spend large 
amounts of money when a large development team waits for test results. 
The alternative is to spend less money and time by considering testing and 
investment in test resources as part of our systems engineering and devel- 
opment planning efforts. The actions recommended in paragraph 1 above 
are the same actions required to address these critical test and evaluation 
needs. 

5. Reducing the time from program initiation to fielding will require the 
combination of all actions suggested above. Further benefits will be derived 
by placing more emphasis on time-certain acquisition. This will be helped 
by better development planning and alignment of incentives. With good de- 
velopment planning, we can assign managers to develop prototypes, critical 
systems or components needed to better understand cost/performance 
trades and reduce risk. It is reasonable to expect that many of these devel- 
opments can be completed in 2-4 years, so one manager will be in place 
from start to delivery. This will help align authority and accountability in 
both government and industry. As these critical subsystems are delivered 
and tested, the risk reduction and domain experience gained in both gov- 
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eminent and industry will allow us to reduce the time required to develop, 
integrate and test the full system. We can also apply meaningful incentive 
programs to link profits to demonstrated performance, and use that per- 
formance as a factor in making future competitive awards. We can rely on 
the experience gained during development planning to apply informed judg- 
ment to adjust requirements to improve value, reduce time, and better esti- 
mate and manage costs. The Department and Congress can assist hy plac- 
ing more emphasis on time-certain acquisition, with the opportunity for 
milestone reviews at the completion of major development planning activi- 
ties. 

I believe action on these five issues will have a significant and demonstrable im- 
pact on our serious acquisition problems. We need to move now with the same ur- 
gency and priority that we expect in combat operations to permit the timely and 
effective development and fielding of new capabilities and services with what I ex- 
pect will be more limited future defense dollars. 

Chairman Levin. Thank you so much, Dr. Kaminski. 

Mr. Adolph? 

STATEMENT OF CHARLES E. (PETE) ADOLPH, CHAIRMAN, DE- 
FENSE SCIENCE BOARD TASK FORCE ON DEVELOPMENTAL 

TEST AND EVALUATION 

Mr. Adolph. Chairman Levin, Senator McCain, members of the 
committee. I’d like to thank you for inviting me today. 

I chaired a recent Defense Science Board (DSB) study of develop- 
mental test and evaluation, and during my opening remarks. I’ll 
summarize the key points from the study. I ask that my written 
testimony, which addresses the major findings and recommenda- 
tions in more detail, be put into the record. 

Chairman Levin. It will be. 

Mr. Adolph. The task force was originally convened in 2007 to 
investigate the causal factors for the high percentage of programs 
completing initial operational test and evaluation in recent years, 
which have been evaluated as not operationally effective and/or 
suitable. 

The task force was asked to assess roles and responsibilities for 
test and evaluation oversight in the Office of the Secretary of De- 
fense (OSD). We were also tasked to recommend changes to facili- 
tate the discovery of suitability problems earlier, and thus, improve 
the likelihood of operational suitability during initial operational 
test and evaluation. 

Very early in the study, it became obvious that the high suit- 
ability failure rates were the result of systemic changes that had 
been made to the acquisition process, and that changes in test and 
evaluation alone could not remedy poor program formulation and 
execution. 

A number of major changes in the last 15 years have had a sig- 
nificant impact on the acquisition process. First, congressional di- 
rection from 1996 through 1999 reduced the acquisition workforce, 
which, of course, includes developmental test and evaluation. In 
many instances, services acquisition organizations went well be- 
yond the mandated cuts, some making up to 60 percent reductions 
in organizations providing acquisition support. 

Concurrent with acquisition reform, the general practice of reli- 
ability growth during development was deemphasized and, in most 
cases, eliminated. This departure from the widely recognized best 
practice may not have been a direct result of acquisition reform. 
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but may instead be related to the loss of key personnel and experi- 
ence, as well as shortsighted attempts to save acquisition funds at 
the expense of increased sustainment and life-cycle costs. Numer- 
ous studies have conclusively demonstrated that investing in sys- 
tem reliability during development will yield a substantial reduc- 
tion in support costs. 

Our study reached the conclusion that the single most important 
step necessary to correct high suitability failure rates is to ensure 
that programs are formulated to execute a viable systems engineer- 
ing strategy, including a robust reliability, availability, and main- 
tainability program, as an integral part of design and development. 

Moving on to government test organizations, in the last 15 years, 
with some exceptions, there’s been a significant decrease in govern- 
ment involvement in test planning, conduct, and execution. One of 
our task force members observed that, in many instances, the gov- 
ernment has gone from oversight to insight to out-of-sight. Our 
task force recommends that government test organizations recon- 
stitute and retain a cadre of experienced test and evaluation per- 
sonnel to perform the test oversight function. 

Regarding OSD roles and responsibilities for test oversight, the 
study team found that the developmental test office, which had ex- 
isted for decades, was disestablished in the late 1990s. Currently 
there is no OSD organization with comprehensive developmental 
test oversight, responsibility, authority, or staff. We recommend 
that the office be reestablished as a direct report to the Deputy 
Under Secretary for Acquisition and Technology, as outlined in the 
proposed legislation. 

I’d like to make a few additional observations about the systemic 
issues that have contributed to the current problems. 

First, during a time of increased programmatic and technical 
complexity, there has been a loss of a large number of the most ex- 
perienced management and technical personnel without an ade- 
quate replacement pipeline. Solutions to acquisition problems must 
begin with reconstituting a trained and experienced government ac- 
quisition workforce, which includes program managers, subject- 
matter experts, as well as systems engineers, contracts personnel, 
testers, and evaluators. 

Second, more attention must be paid to technology readiness, to 
include prototyping and testing crucial technologies. 

Finally, I believe that the major recommendations in the recent 
study chaired by Dr. Kaminski on pre-Milestone A systems engi- 
neer would, if implemented and combined with a revitalized acqui- 
sition workforce, go a long way towards correcting many of the cur- 
rent acquisition problems. 

Thank you. 

[The prepared statement of Mr. Adolph follows:] 

Prepared Statement by Charles “Pete” Adolph 

Mr. Chairman, members of the committee, I am Pete Adolph, the chairman of a 
recent Defense Science Board (DSB) Task Force study of Developmental Test and 
Evaluation (DT&E). I am pleased to present a summary of the study results. The 
findings and recommendations I will discuss represent a consensus of the Task 
Force members and do not reflect an official position of the Department of Defense 
(DOD). 

A DSB Task Force on DT&E was convened in the summer of 2007 to investigate 
the causal factors for the high percentage of programs entering Initial Operational 
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Test and Evaluation (lOT&E) in recent years which have been evaluated as both 
not operationally effective and not operationally suitable. The following are the spe- 
cific issues which the Task Force was asked to assess: 

• Office of the Secretary of Defense (OSD) organization, roles, and respon- 
sibilities for Test and Evaluation (T&E) oversight. Compare organization, 
roles, and responsibilities in both DT&E and OT&E. Recommend changes 
that may contribute to improved DT&E oversight, and facilitate integrated 
T&E. 

• Changes required to establish statutory authority for OSD DT&E over- 
sight. Title 10, U.S.C. has an OT&E focus, and does not address OSD au- 
thority in oversight of DT&E. Recommend changes to title 10 or other U.S. 
statutes that may improve OSD authority in DT&E oversight. 

• Many lOT&E failures have been due to lack of operational suitability. 
Specific problems have been in the materiel readiness sustainment areas of 
reliability, maintainability, and availability. Recommend improvements in 
DT&E process to discover suitability problems earlier, and thus improve 
likelihood of operational suitability in lOT&E. 

PROBLEM DEFINITION 

In recent years, there has been a dramatic increase in the number of systems not 
meeting suitability requirements during lOT&E. Reliability, Availability and Main- 
tainability (RAM) deficiencies comprise the primary shortfall areas. DOD lOT&E re- 
sults from 2001 to 2006 are summarized in Figures 1 through 3. These charts 
graphically depict the high suitability failure rates during lOT&E resulting from 
RAM deficiencies. 
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Figure 3: DoD lOT&E Results for 2006. 


Early in the DSB study, it became obvious that the high suitability failure rates 
were the result of systemic changes that had been made to the acquisition process; 
and that changes in DT&E could not remedy poor program formulation and execu- 
tion. Accordingly, the Task Force study was expanded to address the broader pro- 
grammatic issues, as well as the issues previously identified. 

A number of major changes in the last 15 years have had a significant impact 
on the acquisition process. First, congressional direction in National Defense Au- 
thorization Acts for Fiscal Year 1996, 1997, 1998, and 1999 reduced the acquisition 
workforce (which includes DT&E). Several changes resulted from the implementa- 
tion of Acquisition Reform in the late 1990s. The use of commercial specifications 
and standards was encouraged, unless there was justification for the use of military 
specifications. Industry was encouraged to use commercial practices. Numerous mili- 
tary specifications and standards were eliminated in some Service acquisition orga- 
nizations. The requirement for a reliability growth program during development was 
also deemphasized, and in most cases, eliminated. At the same time, systems be- 
came more complex, and systems-of-systems integration became more common. Fi- 
nally, there was a loss of a large number of the most experienced management and 
technical personnel in government and industry without an adequate replacement 
pipeline because of the personnel cuts. The loss of personnel was compounded in 
many cases by the lack of up-to-date standards and handbooks, which had been al- 
lowed to atrophy, or in some cases, eliminated. It should be noted that Acquisition 
Reform included numerous beneficial initiatives. There have been many programs 
involving application of poor judgment in the last 15 years that can be attributed 
to acquisition/test workforce inexperience and funding reductions. It is probable that 
these problems would have occurred independently of most Acquisition Reform ini- 
tiatives. 

All Service acquisition and test organizations experienced significant personnel 
cuts, the magnitude varying from organization to organization. Over time, in-house 
DOD offices of subject matter experts (who specialized in multiple areas, such as 
promoting the use of proven reliability development methods) were drastically re- 
duced, and in some cases, disestablished. A summary of reductions in developmental 
test personnel follows. The Army essentially eliminated their military Develop- 
mental Testing (DT) component and declared the conduct of DT by the government 
to be discretionary in each program. The Navy reduced their DT workforce by 10 
percent but no shift of “hands-on” government DT to industry DT occurred. The 
trend within the Air Force gave DT conduct and control to the contractor. Air Force 
test personnel have been reduced by approximately 15 percent and engineering per- 
sonnel supporting program offices have been reduced by as much as 60 percent in 
some organizations. The reduction of Acquisition Program Office and Test personnel 
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in the Services occurred during a time when programs have become increasingly 
complex (e.g., significant increases in software lines of code, off-board sensor data 
integration, and systems of systems testing). 

PRINCIPAL FINDINGS AND RECOMMENDATIONS 

Reliability, Availability, and Maintainability 

As a result of industry recommendations in the early 1970s, the Services began 
a concerted effort to implement reliability growth testing as an integral part of the 
development process. This implementation consisted of a reliability growth process 
wherein a system is continually tested from the beginning of development, reli- 
ability problems are uncovered, and corrective actions are taken as soon as possible. 
The Services captured this practice in their reliability regulations, and DOD issued 
a new military standard on reliability, which included reliability growth and devel- 
opment testing as a best practice task. The goal of this process from 1980 until the 
mid-1990s was to achieve good reliability by focusing on reliability fundamentals 
during design and manufacturing rather than merely setting numerical require- 
ments and testing for compliance towards the end of development. 

The general practice of reliability growth was discontinued in the mid- to late 
1990s. This discontinuance may not be a direct result of Acquisition Reform, but 
may be related instead to the loss of key personnel and experience, as well as short- 
sighted attempts to save acquisition funds at the expense of increased life-cycle 
costs. With the current DOD policy, most development contracts do not include a 
robust reliability growth program. The lack of failure prevention during design, and 
the resulting low initial Mean Time Between Failure and low growth potential are 
the most significant reasons that systems are failing to meet their operational reli- 
ability requirements. 

According to Army studies, almost 90 percent of the sustainment costs are directly 
correlated with the reliability of the system. Given the amount of resources con- 
sumed during sustainment, investments in reliability enhancements can provide a 
very large return on that investment. A case study conducted by the Logistics Man- 
agement Institute, provided data that indicated an investment in total program reli- 
ability would yield a substantial reduction in support costs. 

Findings 

• Acquisition personnel reductions combined with acquisition system changes in 
the last 15 years had a detrimental impact on RAM practices 

• With some exceptions, the practice of reliability growth methodologies 
was discontinued during System Design and Development 

• Relevant military specifications, standards, and other guidance were not 
used 

• Suitability criteria, including RAM, were de-emphasized 

• Improved RAM decreases life-cycle costs and reduces demand on the logistics 
system 

• The Deficiency Report can be a valuable tool for early identification of RAM- 
related suitability problems, when used in conjunction with an adequately 
resourced deficiency correction system 

Recommendations 

The single most important step necessary to correct high suitability failure rates 
is to ensure programs are formulated to execute a viable systems engineering strat- 
egy from the beginning, including a robust RAM program, as an integral part of de- 
sign and development. No amount of testing will compensate for deficiencies in RAM 
program formulation. To this end, the following RAM-related actions are required 
as a minimum: 

• Identify and define RAM requirements during the Joint Capabilities Inte- 
gration Development System, and incorporate them in the Request for Pro- 
posal (RFP) as a mandatory contractual requirement 

• During source selection, evaluate the bidders’ approaches to satisfying 
RAM requirements 

• Ensure flow-down of RAM requirements to subcontractors 

• Require development of leading indicators to ensure RAM requirements 
are met 

• Make RAM, to include a robust reliability growth program, a mandatory 
contractual requirement and document progress as part of every major pro- 
gram review 
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• Ensure that a credible reliability assessment is conducted during the var- 
ious stages of the technical review process and that reliability criteria are 
achievable in an operational environment 

• Strengthen program manager accountability for RAM-related achieve- 
ments 

• Develop a military standard for RAM development and testing that can 
be readily referenced in future DOD contracts 

• Ensure a adequate cadre of experienced RAM personnel are part of the 
Service acquisition and engineering office staffs 

Roles and Responsibilities of Government Test and Evaluation Organizations 

The role of the government in the DT process has evolved over the past 50 years. 
With some exceptions, there has been a significant decrease in government involve- 
ment in test planning, conduct and execution, in the last 15 years. 

The traditional role of the government during the DT planning phase included the 
identification of the test resource requirements and government test facilities, the 
development of the test strategy and detailed T&E plans, as well as the actual con- 
duct of T&E. When a program moved from the planning phase to the test execution 
phase, the government traditionally participated in test conduct and analysis; per- 
forming an evaluation of the test results for the program office. With some excep- 
tions, this is no longer the case. Until recently, it was recognized that there should 
be some level of government involvement and oversight even when the contractor 
has the primary responsibility regarding planning and execution of the DT program. 

In addition to the reduction in the number of government acquisition and test per- 
sonnel, the experience level of both government and industry personnel has steadily 
diminished in recent years. A significant percentage of the workforce became eligible 
to retire since 2000, and due to prior downsizing, there has not been a steady pipe- 
line of younger technical personnel to replace them. 

Findings 

The changes in the last 15 years, when aggregated, have had a significant nega- 
tive impact on DOD’s ability to successfully execute increasingly complex acquisition 
programs. Major contributors include massive workforce reductions in acquisition 
and test personnel, a lack of up-to-date process guidance in some acquisition organi- 
zations, acquisition process changes, as well as the high retirement rate of the most 
experienced technical and managerial personnel in government and industry with- 
out an adequate replacement pipeline. 

• Major personnel reductions have strained the pool of experienced govern- 
ment test personnel 

• A significant amount of DT is currently performed without a needed de- 
gree of government involvement or oversight and in some cases, with lim- 
ited government access to contractor data 

Recommendations 

• As a minimum, government test organizations should develop and retain a 
cadre of experienced T&E personnel to perform the following functions: 

• Participate in the translation of operational requirements into contract 
specifications, and in the source selection process, including RFP prepara- 
tion 

• Participate in DT&E planning including Test and Evaluation Master 
Plan (TEMP) preparation and approval 

• Participate in technical review processes 

• Participate in test conduct, data analysis, and evaluation and reporting; 
with emphasis on analysis and reporting 

• Utilize red teams, where appropriate, to compensate for shortages in skilled, 
experienced T&E domain and process experts 

• Develop programs to attract and retain government personnel in T&E career 
fields so that the government can properly perform its role as a contract admin- 
istrator and as a “smart buyer” 

Integrated Test and Evaluation 

Integrated testing is not a new concept within the DOD, but its importance in re- 
cent years has been highlighted, due in part to the growth of asymmetric threats 
and the adoption of net-centric warfare. A December 2007 OSD Test and Evaluation 
Policy Revisions memorandum reinforces the need for integrated testing. Implemen- 
tation of integrated test concepts has been allowed to evolve on an ad-hoc basis. The 
time has come to pursue more consistency in integrated test planning and execution. 
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Collaboration between developmental and operational testers to build a robust in- 
tegrated test program will increase the amount of operationally relevant data that 
can be used by both communities. DT and Operational Test (OT) planning is sepa- 
rate and this inhibits efforts by the Services to streamline test schedules, thereby 
increasing the acquisition timeline and program test costs. 

Additionally, there is a widely held assumption hy many in the OT community 
that only data from independent OT is acceptable for operational evaluation pur- 
poses. While not all information from DT may be useable by the Operational Test 
Agency to support lOT&E, a significant amount of developmental test data can be 
used to partially satisfy OT requirements. More importantly, an operational per- 
spective earlier in the developmental process has often proven to be a catalyst to 
early identification and correction of problems. 

DOD policy should mandate integrated test planning and execution on all pro- 
grams to the extent possible. To accomplish this, programs must establish a team 
made up of all relevant organizations (including contractors, developmental and 
OT&E communities) to create and manage the approach to incorporate integrated 
testing into the T&E Strategy and the TEMP. 

Findings 

• Service acquisition programs are incorporating integrated testing to a 
limited degree through varying approaches 

• Additional emphasis on integrated testing will result in greater T&E 
process efficiency and program cost reductions 

Recommendations 

• Implement OSD and Service policy mandating integrated DT&E/OT&E plan- 
ning and execution throughout the program 

• Require sharing and access to all appropriate system-level and selected 
component-level test and model data by government DT and OT organiza- 
tions, as well as the prime contractor, where appropriate 

• Integrate test events, where practical, to satisfy OT and DT requirements 

Operational Test Readiness Review 

Each Service has an Operational Test Readiness Review (OTRR) process. Al- 
though it varies from Service to Service, the process generally results in in-depth 
reviews of readiness to undergo an lOT&E event. 

Findings 

• A DOD Instruction requires that “the Service Acquisition Executive shall 
evaluate and determine materiel system readiness for lOT&E” 

• Decision authority is frequently delegated to the appropriate Program Ex- 
ecutive Officer 

• Materiel developer is also required to furnish DT&E report to the Under 
Secretary of Defense for Acquisition, Technology and Logistics 
(USD[AT&L]) and Director, OT&E 

• Shortcomings in system performance, suitability, and RAM are usually identi- 
fied during the OTRR 

• In most cases, the operational test readiness certifying authority is well 
aware of the risk of not meeting OT criteria when major shortcomings exist 

• Because of funding constraints, the low priority given to sustainment, as well 
as the urgency in recent years to get new capabilities to the Warfighter, major 
suitability shortcomings have rarely delayed the commencement of dedicated 
lOT&E 

Recommendations 

• Conduct periodic operational assessments to evaluate progress and the 
potential for achieving pre-determined entrance criteria for operational test 
events 

• Conduct an independent Assessment of Operational Test Readiness prior 
to the OTRR 

• Include a detailed RAM template in preparation for the OTRR 

• Require the Command Acquisition Executive to submit a report to OSD 
that provides the rationale for the readiness decision 

OSD Test and Evaluation Organization 

The Task Force was asked to assess OSD roles and responsihilities for T&E over- 
sight. T&E has been a visible part of OSD since the early 1970s, reporting to the 
Research and Engineering command section when it was in charge of acquisition 
oversight and subsequently to the Under Secretary of Defense for Acquisition (now 
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AT&L). The early T&E office was responsible for all T&E, ranges, resources over- 
sight, and policy. In 1983, Congress established an independent Director, OT&E or- 
ganization, reporting directly to the Secretary of Defense (SECDEF), responsible for 
OT&E policy, budget review, and assessments of operational effectiveness and suit- 
ability. The Live Fire Test (LFT) oversight function was created and added to the 
DT&E office responsibilities in the mid-1980s. Later, the LFT oversight function 
was moved to the DOT&E organization. 

In 1999, the DT&E organization was disestablished. Many functions were moved 
to DOT&E, including test ranges and resources, and joint T&E oversight. Some of 
the remaining T&E personnel billets were eliminated to comply with a congression- 
ally mandated (AT&L) acquisition staff reduction. The residual DT&E policy and 
oversight functions were separated and moved lower in the AT&L organization. 

A 2000 DSB Task Force Study on Test and Evaluation Capabilities recommended 
that DOD create a T&E resource enterprise within the office of the DOT&E to pro- 
vide more centralized management of T&E facilities. This recommendation ulti- 
mately led to removing the test ranges and resources oversight from DOT&E, aban- 
doning the notion of centralized management, and the establishment of the Test Re- 
source Management Center (TRMC) in AT&L (as directed by the National Defense 
Authorization Act for Fiscal Year 2003). 

Findings 

Current policy as of December 2007 mandates that developmental and operational 
test activities be integrated and seamless throughout the system life cycle. There 
must be enough experts in OSD with the ability to understand and articulate les- 
sons learned in early testing and the ability to execute the new T&E policy. That 
policy is to “take into account all available and relevant data and information from 
contractors and government sources” in order to “maximize the efficiency of the 
T&E process and effectively integrate developmental and operational T&E.” 

• Currently there is not an OSD organization with comprehensive DT oversight 
responsibility, authority or staff to coordinate with the operational test office 

• The historic DT organization has been broken up and residual DT func- 
tions were moved lower in organization in 1999, and lower yet in 2002 

• Programmatic DT oversight is limited by staff size and often performed 
by generalists vice T&E experts 

• Recruitment of senior field test personnel is hampered by DT’s organiza- 
tional status 

• Existing residual organizations are fragmented and lack clout to provide 
DT guidance 

• System performance information and DT lessons learned across DOD has 
been lost 

• DT is not viewed as a key element in AT&L system acquisition oversight 

• Documentation of DT results by OSD is minimal 

• Access to models, data, and analysis results is restricted by current practice 
in acquisition contracting, and the lack of expertise in the DT organization 

• TRMC has minimal input to program-specific questions or interaction with 
oversight organizations on specific programs 

• Organizational separation is an impediment 
Recommendations 

• Implementation of integrated and seamless DT and OT will require, at a min- 
imum, greater coordination and cooperation between all testing organizations 

• Consolidate DT-related functions in AT&L to help reestablish a focused, inte- 
grated, and robust organization 

• Program oversight and policy, and Foreign Comparative Test (FCT) 

• Have Director, DT&E directly report to Deputy Under Secretary of De- 
fense, Acquisition and Technology (DUSD[A&T]) 

• Restore TEMP approval authority to Director, DT&E 

• Integrate TRMC activities early into DT program planning 

• Make TRMC responsible for reviewing the resources portion of the TEMP 

• If such an organization is established and proves itself effective, consider as 
part of a future consolidation moving LFT back to its original DT location (this 
would require congressional action and DOT&E concurrence) 

Most of the organizational changes recommended above are within the purview 
of AT&L. The LFT change requires the concurrence of DOT&E and a legislative 
change to Title 10 because of the change in reporting official. All the other rec- 
ommendations can be implemented within current DOD authority. 
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Chairman Levin. Thank you so much. 

I think we’ll try a 7-minute round. I’m not sure there will be 
time for a second round; I think we have a vote scheduled around 
11:30 a.m., if I’m not mistaken. 

Let me start with this question. Mr. Sullivan, you commented on 
the reform bill, the Weapon Systems Acquisition Reform Act of 
2009, which I’ve introduced with Senator McCain. Each of you, in 
a way, has commented on it. Let me start with you. Dr. Gansler. 
Would you comment on the bill, any parts you like, any parts you 
don’t like, any additions you might be able to recommend at this 
time? 

Dr. Gansler. Overall, Mr. Chairman, I thought it was in the 
right direction and important. I think that we have to recognize 
that simply writing a memo, passing a law, doesn’t change the sys- 
tem. Each of the areas that you highlighted, I think are in the 
right direction. What we have to be careful of is not going too far 
in one direction. For example, in the conflict of interest, clearly we 
have to avoid conflict of interest, but we don’t have to go so far that 
we have only people who have no experience in those positions. 
That’s the danger of going too far, in terms of the legislation of it. 

In each of the areas, I think that there’s some clarity that could 
be added, but, in general, I think you’ve gone in the right direction. 
For example, you emphasize systems engineering. I think you need 
to define that as including costs so that there’s no ambiguity as to 
whether that’s a pure engineering problem or a cost issue, from a 
design perspective. 

I think the importance of test and evaluation that you highlight 
is clearly something that’s very important. It could be emphasized 
without really changing the title of the office. But, nonetheless, 
when I was Under Secretary, I felt it was a very critical piece. But, 
it’s important, when you talk about test and evaluation, that it be 
viewed as a part of a development process. We learn from that test- 
ing. We do it early, as Mr. Adolph indicated, but we also don’t view 
it as a pass-fail final exam because we’re doing spiral development, 
and we’re continuing to learn from the testing as it goes along, and 
some people have tended to think of the test process as a final 
exam. 

In the area of independent cost analysis, it is absolutely essen- 
tial. On the other hand, we have the Cost Analysis Improvement 
Group (CAIG) office, which I used all the time, and I felt it was 
critically important. The problem is, people don’t want to use their 
numbers. If they say it’s realistically going to cost more, and gov- 
ernment and industry want to put in the low bid in order to get 
a program into the overall budget, that’s a management question; 
it’s not a matter of what organization you set up. But, it’s a very 
important function, as you highlighted. 

In terms of getting the combatant commanders involved in the 
requirements process, that was exactly what we intended with the 
Packard Commission and Goldwater-Nichols intended with the es- 
tablishment of the Vice Chairman of the Joint Chiefs. That was the 
purpose of that statement. Somehow that hasn’t gotten the in- 
tended strength, it has been much more the suppliers than the 
users, if you will, the warfighters. I think it is important to get the 
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combatant commanders much more into the loop on the require- 
ments process, and I think you’ve gotten that properly emphasized. 

I think, in the acquisition area I would emphasize that projected 
unit costs are a military requirement because that’s one of the 
things we’ve lost in some of the programs. Global Hawk started out 
that way, got off track. You mentioned the JSF. That started off 
as a unit cost being one of the requirements for globalization of 
that program. It got out off track because it lost sight of that unit 
cost as being one of its principal considerations in design. 

As I mentioned, in terms of competition, I think it’s really impor- 
tant that we view competition as an option throughout the pro- 
grams, at the prime and at the lower tiers, but not as a law. You 
don’t compete for its own sake; you compete when the current con- 
tractors, prime or sub, aren’t getting better performance at lower 
cost. That’s their incentive for doing it. If you tell them we’re going 
to compete it anyhow, they have no incentive. If you tell them that 
if they can get higher performance and lower cost, then, in fact, 
they will try to achieve that so that they don’t have to compete in 
the next round, and it’s doing exactly what the government wants 
them to do. 

In the same way, if they achieve the objectives, we ought to use, 
in effect, the same way that the commercial world does: through 
price elasticity. If you get a lower cost, we’ll buy more of them. We 
don’t take the money away and put it in the general treasury, so 
there will be a need to create incentives for industry, as well as 
government, for doing a better job of achieving higher performance 
at lower cost, continuously. 

In terms of conflict of interest, I think we need to focus on some 
structural ways to address conflict of interest. We do this, in terms 
of foreign ownership, through limited liability corporations, in ef- 
fect, the special boards are set up. Maybe there’s some ways we can 
do that in order to address conflict of interest without the sort of 
blanket requirement that someone in an engineering job shouldn’t 
know anything about that job. That’s wrong. 

Those are the suggestions that I would have. But, overall, I think 
you are definitely going in the right direction with the bill. 

Chairman Levin. Thank you. We would welcome any specific 
language changes you would recommend. 

Now, Mr. Sullivan, you already commented. Do you have any ad- 
ditional comments? 

Mr. Sullivan. Yes, just briefly. I would say that, given the pack- 
age that we’ve looked at, we support everything in it. We thought 
it was very well-targeted to the key problems on acquisition pro- 
grams, and that the idea to give more authority to the combatant 
commanders, in terms of getting urgent needs met, was a good pro- 
vision. 

I think the most important thing on an acquisition program is, 
at the outset — we’ve all discussed it here — to have more knowledge 
about the requirements that you’re going to build to, before you 
begin, than they have now. 

So, the two provisions that we think are most important — first 
is the cost-estimating provision. We wrote that we don’t see any 
reason why the CAIG actually couldn’t fulfill that position. The im- 
portant thing there is to probably provide the CAIG with more re- 
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sources so they can do their jobs on a more regular basis and rath- 
er than just periodically, at every milestone. The other critical 
thing there is that if the director was moved up, out of the bu- 
reaucracy a little bit more, the estimates that the CAIG make 
might be looked at as less personality-driven, if you will, depending 
upon who’s in office. 

As Under Secretary of Defense, Dr. Gansler used the CAIG, oth- 
ers may not use it as much. If they’re reporting to a higher level, 
and they owe Congress a report, I think that’ll really improve their 
authority and their visibility and their independence. But, the key 
is always going to go back to the requirement-setting process, and 
jelling that with cost estimates. If you begin with not enough infor- 
mation about what it is you’re going to, what you want to build, 
you are not going to get a good cost estimate. So, it really has to 
be based on k^nowledge. 

So, the cost-estimating provision, I think, is very good. The sys- 
tems-engineering provisions that you have in there go a long way 
to providing the knowledge that the cost estimators would need up 
front. 

Chairman Levin. Thank you. 

Dr. Kaminski, do you have any comment? 

Dr. Kaminski. Yes, sir, I do. I made some specific comments in 
my statement, but let me make a couple of big-picture comments, 
if I may. 

If there’s a direction in which I would try to move language in 
the proposed legislation, it would be to focus, not only on process, 
on oversight mechanisms, but a focus on people; people that make 
this system work. I don’t care how good a process you put in place, 
if you don’t have people who are experienced and know what 
they’re doing, I think you’re going to end up with problems. 

When I say “people,” I am referring very broadly to our acquisi- 
tion workforce. Think about this for a minute. These people and re- 
quirements development skills are also key for those who are doing 
development of requirements. They have to participate in this 
tradeoff process to consider the cost of what they’re asking for and 
how those tradeoff with the performance capabilities that are de- 
sired. 

Testing is a critical piece of this process. If you think about it — 
I know my experience is, there isn’t any program I’ve ever worked 
on in which I didn’t know a heck of a lot more about the program 
6 months or a year into it than I did when I started. So this is a 
continuing learning process as we find out things from testing — 
what’s hard to do, what’s easier to do. We need to have a con- 
tinuing dialogue in this requirements tradeoff loop. 

So, requirements developers and acquisition personnel have to 
have training and experience in these systems engineering tools 
and techniques. We would never let a fighter pilot get into an air- 
craft without a very extensive training program to prepare him or 
her for that operation. It’s one of the reasons we do so well with 
our forces. When I compare our requirements for operational re- 
quirements with the training and education requirements of people 
going into acquisition requirements generation, they pale in com- 
parison. We have to be able to develop the training, education, and 
the domain experience that go with this to make it work. 
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I also believe it is worth it for us to look very hard at: How can 
we systematically reduce this long development time? That’s just 
killing us. There are programs in which we can do time-certain ac- 
quisition. Time is money, here. By doing good development plan- 
ning to be able to reduce the risk, following the example of Apollo, 
following the example of our ICBM programs, I believe we can com- 
press that time. But, we need some targets, and some incentives 
to do that. 

We also need to make better and more extensive use of proto- 
types, in a sensible way. Competitive prototypes for some pro- 
grams, perhaps single-thread programs for others. We also need to 
realize that one size does not fit all, here. For example, in dealing 
with lEDs, we find that we need to have an acquisition system 
whose cycle time is measured in weeks. That’s a different acquisi- 
tion system than you need for a strategic bomber. The acquisition 
system has to fit the cycle of the threat that we’re dealing with, 
so we have to tailor it in that way. 

The last comment I would make is that, with respect to imple- 
mentation, what better place could we find to start to implement 
some of these processes, procedures, and people development than 
in these urgent programs that are doing rapid acquisition. Why 
start there? One, it’s urgent. Two, we’ll be able to see the impact 
of changes that we make more quickly in programs that are oper- 
ating in cycle times of weeks or months, and see what’s benefiting 
us and what is not. I’d like to see that commitment to implementa- 
tion because it’ll happen much more quickly than if we simply 
write new processes or new 5000 series in the DOD. 

Chairman Levin. Thank you. 

My time’s way up, but, Mr. Adolph, do you have a brief comment 
on the bill? Any changes? 

Mr. Adolph. Very brief comment. In general, I believe that the 
proposed legislation will go a long way toward alleviating the major 
acquisition problems which have occurred in recent years. 

I also have a quick comment regarding the legislation pertaining 
to the director of the developmental test function. I would add one 
responsibility, that the director participate in the acquisition pro- 
gram reviews conducted by the Under Secretary for Acquisition 
and submit a status of the developmental testing for the programs 
under review. That was the norm in the past, for many years, and 
I think that’s important. 

Chairman Levin. Thank you so much. 

Senator McCain. 

Senator McCain. Thank you, Mr. Chairman. Again, thanks, to 
the witnesses. 

I think there’s general agreement on the part of the witnesses 
that there’s been a dramatic erosion in acquisition workforce and 
test and evaluation personnel. Maybe I could ask you. Dr. 
Kaminski, what happened? 

Dr. Kaminski. I think a number of things happened. Just as 
there’s no one silver bullet, there’s no single entity or action to 
blame. But, let me list some of the things that happened. I think, 
as we looked at major programs of acquisition reform, we tried to 
do more with less. We also had some pretty strong direction from 
some portions of Congress on this. I can recall the chairman of a 
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key committee who publicly made statements that said we had too 
many “shoppers” in the Department, and we really needed to re- 
duce this acquisition workforce. So, over a period of 3 or 4 years, 
nearly 50 percent of the acquisition workforce was taken out. Now, 
the Department was a partner in that, agreeing to those reduc- 
tions. 

Senator McCain. We removed incentives for people to remain in 
the acquisition workforce, in the form of lack of promotion or career 
enhancement? 

Dr. Kaminski. Exactly. If we are able to attract and train the 
right system engineers — if that system engineer doesn’t see some 
path for advancement in the DOD, he or she is going to go find a 
place where they can make a difference and have an opportunity 
for advancement. 

Senator McCain. Which means a revolving door evolves? 

Dr. Kaminski. Yes, sir. But, that revolving door actually may be 
the commercial industry. Again, I return to the comment I was 
making about the importance of cycle time in the acquisition sys- 
tem. If you have someone who’s really worth their salt and able to 
make contributions, and they get into a DOD acquisition system 
which is going to produce something in 15 or 20 years, it won’t take 
them long to realize that their knowledge base is going to erode so 
that they will no longer be valuable to commercial industry, which 
is producing things in 2 - or 3-year timeframes. 

So, to be able to have some revolving door from commercial in- 
dustry back to DOD will benefit from getting these cycle times 
down. 

Senator McCain. Dr. Gansler? 

Dr. Gansler. Yes, Senator McCain. In my prepared testimony, 
I actually showed a graph that came out of the commission that I 
ran. I was shocked at how much the acquisition workforce has been 
allowed to deteriorate. In fact, as we came out of the Cold War, the 
procurement budgets dropped, the number of acquisition people 
came out with it. Then, as Dr. Kaminski said, there was a legisla- 
tive mandate in 1996 to take another 25 percent out. So we’ve 
ended up, now, dramatically, where we had about 500,000 people 
in 1990, we now have about 200,000 people. But, the acquisition 
dollars have gone up dramatically, so you have this huge gap be- 
tween the dollars and the people. 

But, much more important is the point that you just made about 
the officers and the senior people. In 1990, the Army had five gen- 
eral officers; in 2007, when we did the study, there were zero gen- 
eral officers with a contracting background. In the contract man- 
agement organization, the Defense Contract Management Agency 
(DCMA), they had 25,000 people in 1990, they have 10,000 today. 
There’s basically an undervaluing. They used to have four general 
officers; they have zero. So, as you suggest, if you’re a young major, 
you’re not getting into that career field, and, as a result, it’s been 
just totally undervalued. Without those experienced senior people, 
both civilian and military, they don’t know what questions to ask, 
and they are not going to be able to make the right judgments. 

Senator McCain. Obviously, we need to have some personnel pol- 
icy changes, as well. Mr. Sullivan, the issue of Nunn-McCurdy — 
when it was first passed, we thought it was really important and 
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effective. For a while, it was. I think a breach of Nunn-McCurdy 
was a big deal. Now it seems to be a routine kind of event that the 
notification comes over, we see it, and ho-hum. Are we in danger 
of experiencing the same thing with this measure? 

Mr. Sullivan. We have noticed that the Nunn-McCurdy 
breaches we see take a lot longer to resolve and come out with a 
new program, and the funding is continuing on that program as 
they do that. 

As I read the proposal that the committee has now, I think it’s 
really good to have a termination criteria like that. I think GAO 
thinks that’s a good thing. In other words, a program termination 
would not just have someone do a review, but also probably look 
at what happens, what the triggers are for that. I think, right now, 
it states that the program cannot change the scope of work, cannot 
start new contracts, and there might be one other thing that the 
program can do. I don’t know if legislation can basically say that 
that program can no longer obligate money, but that would, to me, 
be a much more direct way to get the point across. If a program 
is automatically terminated when it passes a threshold, and cannot 
obligate money, that might get people’s attention. 

Senator McCain. I can imagine the blowback when some vital 
program is shut down because of our failure to act, but I think 
we’re in agreement that there has to be a more robust oversight 
and ability to exercise that oversight as we see these costs spiral 
completely out of control. 

Mr. Sullivan. Yes, sir. 

Senator McCain. Mr. Adolph, on the issue of Nunn-McCurdy, do 
you think it’s sufficient measure to impose more discipline on the 
cost overruns? 

Mr. Adolph. I believe it’s sufficient. But, the issue that concerns 
me, and the remedy has already been discussed, is to get the tech- 
nology readiness right, at the outset; and second, to get a realistic 
cost estimate. There’s too much concurrency, there are actions that 
are taken in the program to kick the can down the road on Nunn- 
McCurdy, which, in the long run, in my opinion, adversely impact 
the program. The example from the F-35 is getting rid of two of 
the test articles. 

Again, I believe Nunn-McCurdy is fundamentally sound. The 
problems in recent years are a combination of issues which I be- 
lieve the proposed legislation, if it’s really implemented in the Serv- 
ices — and, again, back to Dr. Kaminski and Dr. Gansler’s point — 
in order to implement it, we have to reconstitute the acquisition 
workforce. That’s the first step. 

Senator McCain. Also I think it would be important for us to 
have the combatant commanders more involved in the require- 
ments process, as well. I think sometimes we have neglected that 
aspect of the equation. 

I thank you, Mr. Chairman. I thank the witnesses. 

Chairman Levin. Thank you. Senator McCain. 

Senator Akaka. 

Senator Akaka. Thank you very much, Mr. Chairman. 

I want to add my welcome to the witnesses before us today, and 
to point out that I’m glad that, Mr. Chairman, you are holding this 
hearing on acquisition. For me over the years. I’m beginning to feel 
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that we need to change, or try to change, the heart and soul of 
DOD, and, really, the culture of DOD, and to get to acquisition. 

Gentlemen, as you well know, the reform of DOD acquisition 
process is an extraordinarily complex undertaking involving many, 
many moving parts. However, I believe the first step to tackling 
any problem is to prioritize. So, let me ask this question to each 
of you. In order to most effectively reform the DOD acquisition 
process, what do we need to focus on first? 

First, Dr. Gansler. 

Dr. Gansler. I believe it’s the people. I think if we’ve under- 
valued the importance of this area, in terms of promotion, in terms 
of experience, in terms of numbers, all across the board, both civil- 
ians and military, that we’re not going to get there, even if we pass 
all the laws in the world. We need the people who are going to be 
driving this process. That is my number-one priority. We have ne- 
glected it and, of course, in the last 8 years we’ve been living in 
a rich man’s world, so money doesn’t matter, and if people overrun 
or they don’t perform, “Let’s spend more money.” Now, that’s not 
going to be the case, and we need people who are smart, experi- 
enced, and competent to run their programs — with flexibility, 
though. They have to make tradeoffs of cost and performance, sys- 
tems engineering kinds of work, test and evaluation, so forth. That 
requires management judgment, and you can’t just legislate that, 
and therefore, you need people with experience to be able to make 
those management judgments. That’s my number one priority. 

Senator Akaka. Dr. Gansler, would you also say that another 
part of that would be inadequate staffing? 

Dr. Gansler. Yes. 

Senator Akaka. From what I gathered, there were positions that 
were not filled. 

Dr. Gansler. Absolutely. I was shocked to find that, in Iraq and 
Afghanistan, only 35 percent of the people that were in their jobs 
were qualified for those jobs, even with the minimal qualifications 
that Dr. Kaminski mentioned. Besides that, most of the positions 
weren’t even filled, and they were almost all volunteer civilians in 
the warzone. We need to be able to get some senior military there, 
as well. So, there’s a great lack of people, numbers, but you don’t 
want just numbers, you also want qualified people. Numbers won’t 
do it. It has to be qualified, experienced people. Some of those by 
the way, can come from industry. You can rotate people from in- 
dustry, without conflict of interest, very easily. That’s what we’ve 
had with many of the people with past experience. We do that in 
DARPA, we bring them in and out, and we’ve done it in other parts 
of the government. I think that people are out there with experi- 
ence; we just have to make sure that we make it attractive to get 
them in these government acquisition positions. 

Senator Akaka. Mr. Sullivan. 

Mr. Sullivan. I think, at a strategic level, one of the biggest 
problems the DOD faces is that it is unable to prioritize what 
weapon system capabilities should be put into programs. They have 
95 weapon system acquisition programs, major acquisitions, that 
are underway right now. That’s up from about 75 or so in 2000. 

There is a tendency to have too many programs vying for the ac- 
quisition dollars that are available to DOD on an annual basis. 
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When that happens, you get a very unhealthy competition, where 
many programs and few dollars drive the requirements-setting 
process, which is stovepiped in many ways by the Services. 

So, the Services are all vying for a solution, they want require- 
ments that are very, very tough to make, so that their weapon sys- 
tem can do the most. They, as a result of that, tend to put in opti- 
mistic cost estimates. The funders, of course, are looking at those 
very optimistic cost estimates that are very heavy on assumption 
and very light on facts, data, and actual costs. The acquisition proc- 
ess begins with the lack of the systems engineering that we’ve 
talked about here today. 

So, you have too many programs chasing too few dollars, with 
business cases that are unexecutable. The whole system is seg- 
regated in such a way with process owners and stakeholders that, 
in a way, it works for everyone. That’s the culture. It’s a perform- 
ance-driven culture, and we all understand that and accept that, 
but there’s also a lot of players involved in the culture that create 
this kind of unhealthy competition at the outset. I think that’s the 
culture change that has to take place in this DOD, and that is al- 
most intractable, when you think about it: the difficulty of chang- 
ing that. I think legislation can go a long way to requiring people 
to do certain things, but, in the end, I agree with Dr. Gansler, it 
falls on the people to have the right principles, if you will, to 
change what this system is really supposed to create. 

Senator Akaka. Dr. Kaminski, with your research and study 
background, do you care to comment? 

Dr. Kaminski. Yes. Let me not repeat because I agree with ev- 
erything that’s been said. Let me put another dimension on the 
people. I, too, would answer: most important is people. But, what 
you have to do is recognize the dynamics that involve people. If 
we’re going to attract our veiy best and brightest people to an ac- 
tivity, I’ve found that the principal incentive usually isn’t money. 
With the salaries we pay military officers and civilians in DOD, we 
are still able to attract very capable people to key jobs. 

What attracts them? What usually attracts them is the ability to 
make a difference, to see that they can have a personal impact on 
a major program, on the security of this Nation. 

So, I want to come back again to my issue about time. When pro- 
grams are taking 15 or 20 years, many of those best and brightest 
people say, “What’s the difference? If I’m not going to see some- 
thing happen for 15 or 20 years, why don’t I go someplace where 
I can make something happen sooner?” If they don’t have any free- 
dom to make decisions and influence things because of excessive 
oversight processes and complexity of the process, they’ll go find 
another place to work. 

I want to share with you a perspective that I got from a different 
position. Rather than a perspective of a previous Under Secretary 
of Defense, I want to share the perspective that I gained when I 
served on Active Duty in the Air Force. I’d say I spent two-thirds 
of my career there working on special access programs, part of that 
career, in the early days of the National Reconnaissance Office, 
where I was a program manager for one of our National Reconnais- 
sance spacecraft that’s up and flying today, and for several years 
in the stealth program. Let me just pick one example there, the 
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F-117 program. I was heavily involved in that program. When we 
initiated that program, our plan was, from the beginning of full- 
scale development to field the aircraft in 3 years. We missed that, 
sir. We missed it by a year. We fielded the aircraft in 4 years. Ev- 
erybody who worked on that program could see that they were 
making a difference. They could see that capability coming along 
to be fielded. There was excellent interation and tradeoffs in that 
process, between the testers and the program managers and the 
users who were going to use that aircraft. We made continual ad- 
justments. So, that motivation of people and reduced time go to- 
gether. To attract the right kind of people, we have to work on this 
time issue. 

Senator Akaka. Mr. Adolph? 

Mr. Adolph. As I mentioned in my earlier remarks, people are 
certainly the highest priority, and that’s been reiterated here by 
the previous comments. 

Another area for improvement is certainly overly ambitious re- 
quirements. We need to continue to push technology, without a 
doubt. But, at the same time, we need to ensure that the tech- 
nology is sufficiently matured to incorporate in a weapon system. 
That means prototyping and testing, and testing the prototype item 
in an environment in which it’s going to be placed in a combat en- 
vironment. So, again, first, people. Second, getting the require- 
ments right and making sure that they’re not overly ambitious. 
Again, Dr. Kaminski’s study really addresses the latter issue quite 
well. 

Senator Akaka. Thank you very much, Mr. Chairman. 

Chairman Levin. Thank you. Senator Akaka. 

Senator Collins. 

Senator Collins. Thank you. Thank you, Mr. Chairman. 

Mr. Chairman, I first want to salute you and Senator McCain for 
your initiative. I think you’ve brought forth a bill that will really 
make a difference. I also want to point out that I’m very pleased 
to hear the witnesses today all stress the importance of the acquisi- 
tion workforce. This is an area that Senator Lieberman, Senator 
Akaka, and many of us, have said is the number-one problem, over 
and over again. When I brought this up at the White House Con- 
ference last week, however, some of my colleagues felt that it was 
a lesser problem. So, I was very pleased to see the panel of experts 
before us list this as perhaps the greatest problem that we’re fac- 
ing. 

Dr. Gansler, you and I also worked together, many, many years 
ago, on how to increase competition in Federal contracting, and it’s 
very good to see you here again, as well. 

I want to ask the panel about some of the specific provisions in 
the Levin-McCain bill. In particular, this bill would require that 
costs be considered right up front, when the requirements are set. 
That is a pretty dramatic change from how military requirements 
are set now, when they’re done in an environment that does not 
consider costs, but, rather, an idealized world, where costs would 
not be a factor. 

Dr. Gansler, you endorsed including cost as a design military re- 
quirement right up front, so I’m going to skip over you for this 
question and go to the rest of the panel and ask all of you: Should 
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costs be considered up front when military requirements are first 
established? 

Mr. Sullivan? 

Mr. Sullivan. I think that is a good idea. Dr. Gansler said that 
should be one of the key performance parameters on any major 
weapon systems, in their business case, and I think we would agree 
with that. 

When you’re trying to set requirements, and you’re doing the re- 
quirements analysis that’s needed, I think you begin, at first, in an 
unconstrained manner, and try to get from the user what the user 
would like, in an unconstrained environment. But then it’s critical, 
at some point, to start bonding that with the realities of the time 
it’s going to take to get that to the user, the amount of money, and 
the technologies you have available to do it. So, I don’t think you 
could do that without precluding the ability to think, uncon- 
strained. But the exit criteria would be something that is con- 
strained, at least in a cost range. Then once it exits the require- 
ments process, there is a stage before it would become a set busi- 
ness case and begin as an acquisition program, where that cost 
range could be further reduced to more of a point estimate by con- 
tinuing to make trades. 

Senator Collins. Dr. Kaminski? 

Dr. Kaminski. Senator Collins, I also agree that cost should be 
an upfront factor. I’d add another factor to go with it, though. 

Senator Collins. Yes? 

Dr. Kaminski. Back to my comment about time. Time and money 
go together. When we’re doing requirements tradeoffs, if we’re 
going to have an acquisition cycle time that operates within the 
threat cycle time, those developing requirements have to look at 
the time they want something fielded, as well. So, that needs to be 
an important consideration in this process, and we need to manage 
the time. 

The one other comment I wanted to make with respect to cost 
estimates: we talked earlier about breaches in Nunn-McCurdy. The 
surveillance system that finds the breaches, I think, is fine. But, 
one of the things we want to do is look at root causes. What’s caus- 
ing us to get into Nunn-McCurdy? One of the constructive uses for 
the independent cost estimate that was in the bill might be to add 
another consideration. We’ve talked about the importance of the 
upfront work in systems engineering and development planning. 
One of the things that would be useful for us all to ensure is that 
there is adequate funding at the beginning of a program, between 
Milestone A and B, for us to apply our systems engineering and de- 
velopment planning capabilities to get a good handle on what those 
cost estimates are, and do a thorough job involving stakeholders, 
the requirements part of the equation, and the program manager 
in that process, along with the CAIC. 

I, like Dr. Cansler, used the CAIC very heavily. In fact, I advo- 
cated that we fund programs to the CAIC estimate, but we hold 
the program manager and the contractor to the estimates they de- 
veloped, so we had some finance reserve between the two esti- 
mates. 

Senator Collins. Thank you. 

Mr. Adolph? 
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Mr. Adolph. Senator Collins, certainly we need to consider the 
cost issue upfront. Another very important driver early on is tech- 
nology readiness. In program after program after program, we’re 
into full-scale development and discover that some of the critical 
technologies simply aren’t mature enough. So the program is de- 
layed, and that drives the schedule, and these slippages occur. 

A key to getting the costs right is to ensure that the technology 
is really sufficiently mature. 

Senator Collins. Thank you. 

Dr. Gansler, an issue that hasn’t been discussed today is the im- 
pact of a lack of stability and predictability on a program’s cost 
growth. That obviously can drive up the unit cost. When the mili- 
tary starts out with one plan for acquiring a weapon system, and 
then switches direction or reduces the number of units, doesn’t that 
also drive up the cost? 

The reason I bring this up is, we talk a lot about the errors made 
by contractors, we talk about weapon systems that get gold-plated 
because additional requirements are added, but there’s also an im- 
portant issue, as far as the stability of funding and the lack of pre- 
dictability driving up the unit cost. Could you address that issue? 

Dr. Gansler. Yes. But, let me just briefly comment on your first 
question, though, because frequently the military doesn’t think that 
cost is a military requirement. What they neglect is the fact that 
numbers are a military requirement. If you’re resource-constrained, 
the total dollars that you have divided by the unit cost gives you 
the numbers. Numbers really matter in military operations, wheth- 
er it’s by Lancaster’s law of N-squared or by numbers. Either way, 
numbers really matter, and therefore, cost really matters. That’s 
why it’s so important to have the unit cost as part of the require- 
ments. 

Now you get to your changes, and unless we estimate the cost 
of those changes and their impact on the ultimate cost of the equip- 
ment, we let things get out of hand. So, if cost is a requirement, 
then every time a change comes in, and as Senator McCain said 
earlier, it was 75 a week on the LCS, you have to price those each 
out to make sure it is not having a big cost impact on the program. 

Then, when programs, in general, become relatively stable, you 
don’t get this ripple effect through the budget, which is the point 
that you’re really making. Senator. If I want to pay for program A, 
I take it out of program B, not recognizing that program B now is 
in really bad shape because they don’t have the stability of the 
funding. That stability of funding is a critical issue. 

Senator Collins. Thank you. 

Chairman Levin. Thank you. Senator Collins. 

Senator Burris. 

Senator Burris. Thank you, Mr. Chairman. 

This is an interesting experience for me, to hear the distin- 
guished testimony, Mr. Chairman. I would rather listen than to 
talk. I yield my time. 

Chairman Levin. Thank you. 

Senator Burr is next on this side; he’s not there. 

Senator Chambliss. 

Senator Chambliss. Thank you, Mr. Chairman. 
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Gentlemen, it’s good to see all of you, and thanks for your service 
over the years, and thanks for your being here on what I think is, 
if not the most important issue at the Pentagon today, certainly it 
ranks up pretty high. With these budget times that we’re in, trying 
to figure out a way to buy the weapon systems that we need within 
the timeframe we need is virtually impossible. That’s why your tes- 
timony is so critical. 

I want to go back to what Senator Collins was talking about 
there, and what all of you have alluded to in some part, and that’s 
this issue of instability, whether it’s requirements, whether it’s per- 
sonnel, whether it’s funding, or whether, as you say, Mr. Adolph, 
it’s the technology aspect of it, where we, too many times, tend to 
come up with a great idea, and by the time we get into the produc- 
tion phase, we’ve wasted, not only time, we’ve wasted money, but 
oftentimes, there’s a new idea that has been developed in the 
meantime. 

I’m amazed. Dr. Kaminski, that you say, on the F-117, that we 
were in production in 4 years. Gosh, if we had done that with the 
F-22, we wouldn’t be having the arguments we’re having today, 
and we’d have a great airplane, and we’d be worrying about other 
issues. 

But, it doesn’t make any difference whether you’re talking about 
an aircraft carrier, whether you’re talking about FCS or a tactical 
fighter, we come up with this idea, and we get into the R&D phase, 
and there is, all of a sudden, a great idea, but instability in all four 
of those areas runs that cost up tremendously. Then, you throw in 
what Senator Collins alluded to, about the number of these units 
that we’re going to buy, and all of a sudden it explodes again and 
it becomes such a negative at the Pentagon, rather than the posi- 
tive that it started out to be. 

My question for a comment from each of you is: How do we get 
back to this? How do we get back to the point to where we come 
up with this idea? If it’s a tactical fighter, it’s supposed to be air- 
to-air, or supposed to be air-to-ground, whatever it may be, how do 
we develop that and get it into production right away, without 
technology intervening and all of a sudden having to add this and 
add that? How do we get our arms around that issue? 

Jack, let’s start with you. 

Dr. Gansler. I think one of the main opportunities we have is 
to accept the concept of spiral development, that for the block 1 
system, we have a fixed set of requirements, we have a fixed price 
that we’re trying to get a fixed schedule, as Dr. Kaminski said, and 
we go ahead with block 1 under the assumption that if we can then 
demonstrate new technology, if we find that the user needs some- 
thing different, if we find that even the logistician has a problem 
with maintenance of that equipment or the reliability as we deploy 
it, that becomes block 2, block 3, block 4. But, block 1 has to use 
proven technology and get out there quickly, all with a set of con- 
straints. So, it’s a stable program, as you point out. 

The most successful acquisition that, in fact. Congressman Aspin 
used to always highlight, was the Navy’s Polaris, Poseidon, Tri- 
dent. When I was in industry, I always knew how much money I 
was going to get next year for that program. I could hire, I could 
plan my workforce, and so forth. That stability is very important 
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for efficiency. I think, if we go to a spiral development model, 
whereby block 1 is stable, and block 2 is being developed while 
block 1 is being deployed, you have the concept of stability built in, 
and evolutionary systems are still stable. That’s the way the real 
world, the commercial world, works. You constantly are upgrading 
the software, the hardware of computers, but you’re constantly get- 
ting higher performance at lower cost. That has to the be objective 
of each of the blocks as we’re going along in spiral development. 

Senator Chambliss. Mike? 

Mr. Sullivan. Goldwater-Nichols legislation, from years ago, 
tried to bring jointness into the DOD. I think it succeeded on the 
operations side. We now have the combatant command’s matrix, 
the military forces that fight wars jointly very well. The same thing 
did not occur on the acquisition side. I think if you look at what’s 
going on now, there’s a kind of a stovepipe system for how you get 
programs started, and that creates this kind of competition for big 
requirements and cost estimates that are heavy on assumption. 

I would agree with everything that Dr. Gansler just said. If you 
can work on that and get a more joint requirement setting and 
funding system, and try to get the proper balance of weapon sys- 
tems started, try to get rid of the stovepipes, you’d have an envi- 
ronment that could do what Dr. Gansler, I think, is describing, a 
little more easily. 

Another program I would throw out is the F-16 program. Back 
in the 1970s, it was a block program. It was the capabilities that 
the Air Force wanted for the F-16. They knew that they needed an 
aircraft faster than they could develop the technologies to get those 
capabilities, so they had blocks. If you look at the F-16s, and, for 
that matter, the F-15s, performance over the last 30 years, it’s 
pretty impressive. They basically upgraded those aircraft pretty ef- 
ficiently as they went because they started without that big revolu- 
tionary leap, that one-step, big-bang kind of a thing. 

So, we’ve done this before, and I think it’s possible to get back 
to it, but this is where the culture comes in. I think there’s some 
culture change that needs to take place. 

Dr. Gansler has other things that I read in his report that would 
help this significantly. Open systems, for example, on these weapon 
systems, when, if you can make interfaces on the weapon systems 
uniform, you can keep proprietary data that subcontractors have 
that supply subsystems to them, and all they have to do is have 
the proper interface. Then you can open up competition. 

One more point I would make is the difference between tech- 
nology development and product development probably needs to be 
better understood. Technology is the kind of thing you should think 
of when you think of scientists and lab coats and trial and error, 
and it’s done in a smaller-dollar environment, where you can test, 
use trial and error, and make mistakes. You have to keep that off 
of these acquisition programs. I think someone up here, I don’t 
know if it was Dr. Kaminski or Mr. Adolph, said that when you 
have a technology that’s not mature, and it’s on an acquisition pro- 
gram that’s driving towards production, you have an entire work- 
force — an entire supply chain, for that matter — that’s waiting for 
that technology to mature. The burn rate is pretty big on that 
workforce that you have. 
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Dr. Kaminski. I’d emphasize that point. While everybody’s wait- 
ing, we’re paying, and so what you want to do is decouple those 
two. 

In terms of approaches to do this, I agree completely with the 
spiral development approach. One problem I see is in the applica- 
tion of spiral. In the few spirals I’ve seen that we actually imple- 
mented, we had everything but the kitchen sink in the first spiral, 
rather than stretching this out over a period of time, like we did 
in Apollo. So, we need development planning. You have to have a 
plan with stability to do this. The program manager has to have 
the discipline and the experience to reject things that aren’t in the 
plan or that aren’t mature enough to be harvested. 

I have found that one of the characteristics of a good program 
manager is a big lower right-hand drawer in their desk, and what 
went into that drawer were all the ideas for improvement. The 
drawer was kept closed until such time as we fielded the first sys- 
tem; that is the time open the drawer and look to see what devel- 
opment plans we need to deal with shortfalls or upgrades for that 
system. 

This time-certain development is important because, as time goes 
by, the technology gets old, and new ideas are introduced that end 
up being disruptive to the process. So, time is a key factor here. 

This stability issue is really key. If I look back through my whole 
career, there is only one program I ever worked on where we actu- 
ally produced the system at the rate we planned. That was the F- 
117. We built one a month. Every other program I can think of, by 
the time we were done with development, we couldn’t afford the 
build rate that we planned. 

Senator Chambliss. Anything to add, Mr. Adolph? 

Mr. Adolph. Just one thing, since I certainly agree with the 
other panel members, but with my background in tests. I’ll add 
that issue. In the case of the F-15 and F-16, which were men- 
tioned earlier, I worked on both programs, and I was out in the 
field, test business, working for the Air Force. 

Chairman Levin. Please talk a little bit louder, if you would. 

Mr. Adolph. Yes. In the case of the F-15 and F-16, I was work- 
ing for the Air Force in the field at the time, and the F-15 test pro- 
gram was structured well; we had 19 test articles and sufficient ar- 
ticles in the case of the F-16 as well. In the latter F-16 case, the 
propulsion system had been matured. In order to move a program 
along, you need an adequate number of test articles, and you need 
to be able to do as much testing as you can. Avionics is a good ex- 
ample, where considerable testing can be accomplished on a test 
bed platform, rather than the developmental platform. 

So, that’s, I think, one key component of keeping a program mov- 
ing. When a program stagnates, when you only have one or two 
test articles, and you have a ‘standing army’ of test personnel wait- 
ing, and the fixed cost of those people is almost as great as the in- 
cremental cost of doing additional testing, particularly when you 
recover the article. That’s not the case in a missile program, where 
the test article is destroyed on each test. 

Chairman Levin. Thank you. Senator Chambliss. 

Senator McCaskill. 

Senator McCaskill. Thank you, Mr. Chairman. 
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Let me start with Mr. Sullivan, and there have been several ref- 
erences that sometimes we have excessive oversight with no value 
added. That’s a frustrating comment to me, in that I have watched 
the Defense Contracting Audit Agency basically get taken to task 
by their peers for failure to even follow basic auditing standards. 
I’ve read, I can’t tell how many of your reports that have, in fact, 
identified weapons acquisition as high risk, since 1990. It is not as 
though the oversight’s not occurring, it’s just that it’s being ig- 
nored. It’s not adding value because nobody’s paying any attention 
to it. 

Let me ask you, in that regard, about JCIDS, Planning, Pro- 
gramming, Budgeting, and Execution (PPBE), and Future Years 
Defense Program (FYDP). Now, first of all, I think we ought to 
pass a law that they quit talking in initials because you all know 
what those things are I just said, but I guarantee you, nobody that 
I work for in Missouri has any idea what JCIDS is, PPBE is, or 
FYDP. What they are is: one, is all the Services getting together 
and basically giving each other what they want; two, is a 2-year 
calendar-driven process, where they’re supposed to be figuring how 
they’re going to spend the money; and three, it is the Secretary of 
Defense trying to low-ball what it’s going to cost, long-term, in 
order to make sure that the other two go along with it. Is that an 
accurate summary of what those three are? 

Mr. Sullivan. The — that [Laughter.] 

I would say, number one, that the JCIDS is a requirement-set- 
ting process; it’s where all of the Services tend to get together and 
figure out what it is that they require. 

Senator McCaskill. But, in your testimony, Mr. Sullivan, you 
pointed out, they never say no. 

Mr. Sullivan. That’s true. 

Senator McCaskill. Have they ever said no that you’re aware of 
since you’ve been looking at this? 

Mr. Sullivan. We issued a report for this committee, I think, 
about a year ago, where we looked at that, and it was — I think the 
JCIDS process then, if I’m not mistaken — this may not be exact, 
but I can get it for you — about 90 percent of the proposals that 
went in were granted. 

[Excerpt of GAO-08-1060 follows:] 
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Instead, the military services retain most of DOD’s analytical capacity and 
resources for requirements development. The Joint Staff recently initiated a 
project to capture the near-, mid-, and long-term needs of the services and 
other defense components, and to synthesize them with the needs of the 
COCOMs. However, DOD officials told us that determining how best to 
integrate COCOM and service capability perspectives will be challenging 
because of differences in roles, missions, and time frames. Efforts have also 
begun to streamline the process and reduce the time it takes to validate 
proposals. 

j Unitad States Government Accountability Office 


Senator McCaskill. Explain to me, so I understand, what is the 
makeup of the three groups of people. These three are really the 
stovepipes that cause a lot of the problem because you have, 
“Here’s what we want, here’s how we pay for it, and here’s how we 
figure out how much it’s going to cost in the long run.” 

Mr. Sullivan. Yes, ma’am. 

Senator McCaskill. What I don’t understand is: what is keeping 
them from making that one thing, so they all have to do all of that 
at once? 
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Mr. Sullivan. The requirement-setting process, the JCIDS, is 
run by the JROC, and the JROC is made up of the military service 
chiefs, more or less. It’s run by the Vice Chief of Staff. So, it, in 
a sense, is a matrix organization, but it receives most of the pro- 
posals for needs to be validated as weapon systems from the three 
Services, as stovepipes. 

Now, the JCIDS was established to have something called “func- 
tional capability boards,” which were supposed to be a matrixed or- 
ganization based on looking at things like battlespace awareness or 
force protection or force projection, looking at it functionally in- 
stead of across the Services. What we found when we did that 
study was that the DOD has not staffed those functional capability 
boards properly, it hasn’t resourced them properly, so they don’t 
really do a lot of joint decisionmaking to send proposals forward to 
the JROC. So, mostly what they are receiving is proposals for capa- 
bilities that are coming from the Air Force, the Navy, and the 
Army, and they compete with each other. 

Senator McCaskill. The sense is, at JCIDS, that if you say no 
to what they want, then they’re going to say no to what you want. 
Isn’t that part of the problem, in terms of the way this is actually 
supposed to be oversight with no value added. 

Mr. Sullivan. I would say that the oversight is far from perfect. 
There’s not a lot of value to it. 

Senator McCaskill. The idea of the Joint Staff is that they’re 
supposed to be picking winners and losers. What is your sense of 
how effective the organization of the Joint Chiefs office has been, 
in terms of weapon acquisition and picking winners and losers? 

Mr. Sullivan. In terms of that organization that does that pick- 
ing of the winners and losers, the JROC, there are a lot of 
redundancies right now in the weapon system portfolio because 
they can’t make proper decisions, it seems to us. 

I’ll give you an example. Right now, there are some unmanned 
aerial systems that are in development that we believe, and I think 
the Department actually believes, should have been joint programs, 
but the Services had unique-enough requirements and missions to 
be separate. Right now, there is the Predator, which has been very 
valuable in Iraq and Afghanistan and is an Air Force program. Air 
Force is currently making a bigger, more powerful Predator, called 
the Reaper. Another one of the Services, I believe it’s the Army, 
has started what they call the Sky Warrior. Both drones are done 
by the same contractor and both with very similar requirements, 
but the Services have determined that they’re different enough that 
they each have to have their own acquisition program. That’s the 
sort of thing that the JROC is contending with; it’s a very parochial 
kind of an attitude. 

Senator McCaskill. So, is this not fixable? 

Mr. Sullivan. This is the cultural aspects of this, I think, that 
we’ve all been addressing, where you can write legislation, you can 
have policies. 

Senator McCaskill. It doesn’t do any good. 

Mr. Sullivan. But, unless you change the culture, and I guess 
that is the number one question: How do you change that culture, 
that has been in existence for so long, to try to turn it a little bit, 
to do things a little more efficiently? It’s very much a culture issue. 



67 


Senator McCaskill. I also think, Mr. Chairman, one of the 
things we have to do, in some instances, is take out a mirror be- 
cause I think there are times that, when the military has tried, ei- 
ther by the way they’ve done the budget or by actually being so 
bold as to say we need to wind down a program, that Congress de- 
cides, because of our parochial interests, that it’s important that we 
go to bat to augment the budget to take care of the weapon system 
that we think is important in our part of the world. So we con- 
tribute to this problem, and I think we shouldn’t complete this 
hearing without at least acknowledging that sometimes Congress 
has their hand in this stew. 

Mr. Sullivan. Yes, ma’am. I would say, sometimes we should 
think of this as a system that is in equilibrium because there are 
very many stakeholders in this system that are getting specific 
things, even the GAO. It’s a pretty good employment program for 
us. We report on cost schedule and performance problems, and we 
have been doing it 30 years. So, culturally speaking, if you examine 
it as a system that, maybe, is in equilibrium, in a sense — it’s not 
necessarily broken for the people that are involved in it. 

Senator McCaskill. Thank you. 

Thank you, Mr. Chairman. 

Chairman Levin. Thank you very much. 

Just one quick comment on that. We do, in this bill, attempt to 
make the JROC system cover some of the issues that Senator 
McCaskill talked about by requiring it to make these early trade- 
offs, by looking at cost and at schedule, by the way, as well as the 
requirements and the performance requirements. 

Mr. Sullivan. Yes, sir. 

Chairman Levin. There is an effort in this bill. Senator 
McCaskill. I just want to give you assurance that at least this bill 
attempts to do what we can, legislatively, to put those elements 
into the JROC process, which would, hopefully, cut down the paro- 
chialism by forcing consideration of cost and schedule, not just re- 
quirements. So, I’d just get that on the record. 

Thank you. Senator McCaskill. 

Senator Martinez is next. 

Senator Martinez. Mr. Chairman, thank you very much. 

Dr. Gansler, I wonder if I could ask of you to define for me a lit- 
tle more broadly what you mention as a “holistic approach” to de- 
fense needs. 

Dr. Gansler. Yes. In fact, when I even looked at the situation 
in Iraq and Afghanistan, I was surprised to find that the State De- 
partment, United Sates Agency for International Development, and 
DOD were all there, but not integrated, in terms of the buying, the 
contracting, and planning purposes. But, much more at a higher 
level than that, it seems to me that the world of the 21st century 
is going to require us to combine hard and soft power in the kinds 
of operations involving expeditionary operations or insurgent oper- 
ations that we get involved with around the world. So, we’re going 
to need a much closer tie between, in that case. State and Defense. 
But, I would go further and say. Homeland Security and Defense. 
It shocked me, in fact, that Saddam Hussein didn’t try and pull 
some terrorist actions at the same time as we attacked him. I 
would expect that’s going to happen in the future. So, it is an inte- 
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grated, holistic perspective that is the meaning of “national secu- 
rity” in the 21st century, that will involve Homeland Security, that 
will involve State, will involve the intelligence community, will in- 
volve the DOD. 

I think that’s basically what National Security Advisor General 
Jim Jones is now trying to do, some of the restructuring that’s tak- 
ing place at the national security level within the President’s office. 
I think this combination of soft and hard power is going to be re- 
quired, very clearly. That’s what I was thinking of, in terms of the 
“holistic perspective.” But, also the types of threats. Think about it, 
the energy case, the pathogen spreading worldwide, the economic 
crisis that we’re in, these are all national security issues for the 
21st century that we have to start to incorporate into our thinking 
of national security. That’s what I had in mind. 

Senator Martinez. Thank you. I think it is an intriguing future 
that we all are stepping into. I think, by the way, Saddam would 
have if he could have. 

But, the LCS program is one that I’m very much fond of, and one 
that I think is essential to the national security interests of our 
country. I wonder how you believe the Navy got so far off track on 
that particular program? Was it too many requirements being put 
on the platform by the Navy? Was it the length it’s taken to de- 
velop it? We now have two hulls being developed. So, to the extent 
that any of you could speak to the LCS program and what you see, 
going forward, it would be helpful to me. 

Dr. Gansler. I just actually published, I think last week, a DSB 
report which looks specifically at the LCS and the presidential heli- 
copter, a couple of programs of that sort, where the initial concept 
was: get something relatively fast, take something “off-the-shelf’ 
that could be used, and addressing Dr. Kaminski’s point about 
rapid acquisition — then maybe block 2, 3, and 4 would add some 
of the additional things. 

What happened on the LCS: the first thing they said was, “Has 
to go through Sea State 8.” Well, that’s like going through a hurri- 
cane and it wasn’t initially designed for that. Then the next thing 
they said was, “Well, it has to have a new Navy sprinkler system.” 
The sprinkler that was in the system in those two ships that you 
talked about, it wasn’t going to be adequate, for some reason or 
other. So, each of these special requirements ended up basically 
changing the original block 1 system and introduced the instability, 
cost growth, and schedule impact that we’ve talked about in all 
these other programs. 

Yes, we badly needed the LCS, but is it going to have to be a 
battleship? Does it have to do everything that a battleship does? 
How it’s going to be used by the Navy was resisted, in terms of the 
nontraditional solutions. So, this is the culture change that we’ve 
been talking about, as well. If you kept the cost and the schedule, 
and got a block 1 system out there much faster — I like the idea 
that we did it competitively. I think that was a very important 
step. So, I would encourage that to be done in these earlier dem- 
onstration systems. 

Senator Martinez. Dr. Kaminski? 

Dr. Kaminski. Yes, sir. I think the concept initially was a good 
concept. What was missing was the upfront systems engineering 
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and development planning that I spoke about. I can remember 
back to the time I was serving in the acquisition executive’s job in 
DOD; I was a big proponent of commercial practice, and also in 
buying commercial systems where that made sense. But, if some- 
body came to me and said we were going to buy a commercial ship 
for this mission, my first question to him was, “What are we going 
to change in the mission to be able to do it with a commercial 
ship?” If the answer is “nothing,” then I have to ask a second ques- 
tion, “Wait a minute, this commercial ship doesn’t have the kind 
of military requirements you would have for fire-safe cables, for a 
fire sprinkler system, or a whole variety of things. What are you 
going to do about those?” I don’t think we started asking those 
questions about the LCS program until we were well into the pro- 
gram, so we missed this upfront set of tradeoffs. Those are trade- 
offs that you have to make. They can be made sensibly if you ap- 
proach them, understand them, look at the costs, the performance, 
and the schedule to make those tradeoffs. I don’t believe we made 
those tradeoffs upfront. That, for me, does not necessarily damn 
the LCS program. There may still be value derived from looking at 
these tradeoffs and now making sensible decisions to go forward. 
I agree with Dr. Gansler about the advantage of having a competi- 
tive environment to be able to do that. 

Senator Martinez. Mr. Adolph? 

Mr. Adolph. I’m not really familiar with the LCS. I’ve had no 
involvement with the program. 

Senator Martinez. Okay, and you, Mr. Sullivan? I don’t know 
whether you’ve had any experience. 

Mr. Sullivan. I don’t have any specific experience with that, but 
we do have a team that looks at all of our Navy ships, and I’m sure 
we’ve had a report on that recently, which I could submit for the 
record. 

Senator Martinez. That would be great. I’d appreciate that. 

Mr. Sullivan. Okay. 

[Excerpt of GAO-05-255 follows:] 
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Plans Need to Allow Enough Time to 
Demonstrate Capability of First Littoral 
Combat Ships 


Why GAO Did This Study 

To conduct operations in littorals— 
shallow coastal waters — the Navy 
plans to build a new class of 
surface warship: the Littoral 
Combat Ship (LCS). LCS is being 
designed to accomplish its 
missions through systems 
operating at a distance from the 
ship, such as helicopters and 
unmanned vehicles, and that will 
be contained in interchangeable 
mission packages. The Navy is 
using an accelerated approach to 
buy the LCS, building the ships in 
“flights." Flight 0, consisting of four 
ships, will provide limited 
capability and test the LCS 
concept. The schedule allows 12 
months between the delivery of the 
first Flight 0 ship and the start of 
detailed design and construction 
for Flight 1 ships, Estimated 
procurement cost of the Flight 0 
ships is $1.5 billion. 

The Congress directed GAO to 
review the LCS program. This 
report assesses the analytical basis 
of LCS requirements; the Navy’s 
progress in defining the concept of 
operations; the technical maturity 
of the mission packages; and the 
basis of recurring costs for LCS. 


What GAO Recommends 


GAO recommends that the Navy 
analyze the effect of larger surface 
threats on LCS operations, 
incorporate the impact of LCS into 
helicopter force structure, and 
sufficiently experiment with Flight 
0 ships before selecting a Flight 1 
design. The Department of Defense 
partially concurred with GAO’s 
recommendations. 

www.gao-gov/cgi-bin/gelrpt?GAO-05-255. 

To view the full product, including the scope 
and methodology, click on the link above. 

For more information, contact Paul Francis at 
(202) 5l2-2811orfrancisp@gao,goy. 


What GAO Found 

The formal analysis of requirements for U.S. littoral combat operations — 
conducted after the Navy established the LCS program — examined a number 
of options, such as the extent to which existing fleet assets or joint 
capabilities could be used. While the Navy concluded that the LCS remEiined 
the best option, it focused on LCS requirements for combating small boats. 
The Navy did not conduct an analysis of the impact of larger surface threats 
LCS may face. Such threats may increase the risk to LCS operations when no 
other nearby U.S. forces are available to help. 

The Navy has developed both a broad concept and more detailed plans on 
how the LCS will be employed. It has also identified a number of challenges 
that could put the LCS concept at risk, such as manning, logistics, and 
communications. For example, reduced manning — a key goal of the LCS 
program — may not be achievable because maintaining and operating the 
ship’s mission packages, such as the MH-60 helicopter, may require more 
sailors than the current design allows. Further, the Navy has not yet 
incorporated the numbers of helicopters that will be needed to fulfill LCS’ s 
concept of operation into its force structure and procurement plans. If the 
Navy’s efforts to meet these challenges axe not successful, the Navy may not 
have sufficient time to experiment with the Flight 0 ships and integrate 
lessons learned into planning and designing for follow-on ships. 

While the Navy designed the first LCS to rely on proven technologies and 
systems, a number of technologies to be used in LCS’s mission packages 
have yet to be sufficiently matured — that is, they have not been 
demonstrated in an operational environment — increasing the risk of cost and 
schedule increases if the technologies do not work as intended. 

Technologies must also be demonstrated for systems on the LCS seaframe. 
Other factors may affect the availability of mature technologies and 
subsystems, such as making the modifications necessary for adaptation to 
the LCS and transitioning projects from the laboratory to production. 
Collectively, these technology issues pose an additional challenge to the 
Navy’s ability to sufficiently experiment with Flight 0 ships in time to inform 
the design efforts for follow-on ships. 

Procurement costs for the Flight 0 ships remain uncertain. The basis for the 
seaframe cost target — $220 million — spears to be more defined than for the 
mission packages, as the Navy has performed various cost analyses that 
consider the challenges in detailed design and construction. The Navy seeks 
to meet the cost target by trading between capability and cost. Cost data for 
the Flight 0 mission packages are not as firm in part because of the 
uncertainties associated with immature technologies. 
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Senator Martinez. I’m sure going to try to look up your article, 
as well, Dr. Gansler. 

You haven’t written on DGD-1000 and the DGD-51 to date, 
have you? 

Dr. Gansler. Not lately. 

Senator Martinez. Okay. Do any of you share an opinion on the 
needs of the Navy as it relates to these two programs? 

Mr. Sullivan. I would just say that we’re going to come out with 
our annual assessment of programs, and I think hoth the DGD- 
1000 and LCS are going to be programs that are covered in that. 
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so you will get our take on them, probably in the next couple of 
weeks. I’ll follow up, as well. 

Senator Martinez. Very good. Thank you. 

[Excerpts of GAO-09-326SP follows:] 
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Why GAO Did This Study 

This is GAO’s seventh annual 
assessment of selected Department 
of Defense (DOD) weapon 
programs. The report examines 
how well DOD is planning and 
executing its weapon acquisition 
programs, an area that has been on 
GAO’s hi^-risk list since 1990. 

This year’s report is in response to 
the mandate in the joint 
explanatory statement to the 
Consolidated Security, Disaster 
Assistance, and Continuing 
Appropriations Act, 2009. The 
report includes (1) an analysis of 
the overall performance of DOD’s 
2008 portfolio of 96 major defense 
acquisition programs and a 
comparison to the portfolio 
performance at two other points in 
time — 6 years ago and 1 year ago; 
(2) an analysis of current cost and 
schedule outcomes and knowledge 
attained by key junctures in the 
acquisition process for a subset of 
47 weapon programs — primarily in 
development — ^from the 2008 
portfolio; (3) data on other factors 
that could impact program 
stability; and (4) an update on 
changes in DOD’s acquisition 
policies. To conduct our 
assessment, GAG analyzed cost, 
schedule, and quantity data from 
DOD’s Selected Acquisition 
Reports for the programs in DOD’s 
2003, 2007, and 2008 portfolios. 
GAO also collected data from 
program offices on technology, 
design, and itmnufacturing 
knowledge, as well as on other 
factors that might affect program 
stability. GAO analyzed this data 
and compiled one- or two-page 
assessments of 67 we£q)on 
programs. 

To view the full product, including the scope 
and rnethodoiogy, click or GAO-09-326SP. 
For more information, contact Michael J. 
Sullivan at (202) 512-4841 or 
sullivanm @ gao.gov. 


What GAO Found 

Since 2003, DOD’s portfolio of major defense acquisition programs has grown 
from 77 to 96 programs; and its investment in those programs has grown from 
$1.2 trillion to $1.6 trillion (fiscal year 2009 dollars). The cumulative cost 
growth for DOD’s programs is higher than it was 5 years ago, but at $296 
billion, it is less than last year when atfjusted for inflation. For 2008 programs, 
research and development costs are now 42 percent higher than originally 
estimated and the average delay in delivering initial capabilities has increased 
to 22 months. DOD's performance in some of these areas is driven by older 
programs, as newer programs, on average, have not shown the same degree of 
cost and schedule growth. 


Anahrete nf Hon Muirtr OefArtse Acoul^tion Prooram Portfolios (Fiscal Year 2009 Dollars) 

Portfolio status 

Fiscal year 
2003 portfolio 

Fiscal year 
2007 Dortfollo 

Fiscal year 
2008 portfolio 

Number oi oroarams 

77 

85 

96 

Total Dtanned corrunitments 

$1.2 trillion 

$ 1.6 trillion 

$1.6 trillion 

Commitmenls outstandina 

$724 billion 

$875 billion 

$786 billion 

Change to (oial research and development costs 
from first estimate 

37 percent 

40 oercent 

42 oercent 

Chanoe In total acauisitlon cost from first estimate 

19 oercent 

26 oercent 

25 oercent 

E.sllmated total acauisitlon cost orowth 

$163 billion 

$301 billion* 

$296 billion 

Snare of programs with 25 percerti or more 
increase in brooram acauisitlon unit cost 

41 oercent 

44 oercent 

42 percent 

Averaoe delav in delivenna initial caoabiiiiies 

18 months 

21 months 

22 months 


Ssuree: SAO enelysaso' OOO Osts. 

'Last year. GAO reported total acquisilion cost growth for the fiscal year 2007 portfolio was 
$295 billion in fiscal year 2008 dollars. This figure is now expressed in fiscal year 2009 dollars. 


For 47 programs GAO assessed in-depth, the amount of knowledge that 
programs attained by key decision points has increased in recent years; but 
most programs still proceed with far less technology, design, and 
manufacturing knowledge than best practices suggest and face a higher risk of 
cost increases and schedule delays. Early system engineering, stable 
requirements, and disciplined software management were also important as 
programs that exhibited these characteristics experienced less cost growth 
and shorter schedule delj^ on average. Program execution could be hindered 
by workforce challenges. A majority of the programs GAO assessed were 
unable to fill all authorized program office positions, resulting in increased 
workloads, a reliance on support contractors, and less personnel to conduct 
oversight 

In December 2008, DOD revised its policy for major defense acquisition 
programs to place more emphasis on acquiring knowledge about 
requirements, technology, and design before programs start and maintaining 
discipline once they begin. The policy recommends holding early systems 
engineering reviews; includes a requirement for early prototyping; and 
establishes review boards to monitor requirements changes — ail positive 
steps. Some programs we assessed have begun implementing these changes. 
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Common Name: DDG1000 


DDG 1000 Destroyer 

The Navy’s DDG 1000 destroyer (formerly known as 
DD(X)) is a multimission surface ship designed to 
provide advanced land attack capability in support 
of forces ashore and contribute to U.S. military 
dominance in littoral operations. The program 
awarded contracts for detail design in August 2006 
and negotiated contract modifications for 
construction of two lead ships in February 2008. The 
program will continue to mature its technologies 
and design as it approaches construction start, 
currently planned for February 2009. 
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Four of 12 DDG 1000 critical technologies are fully 
mature, having been demonstrated in a sea 
environment. Six other technologies are 
approaching maturity, but 5 of them will not 
demonstrate full maturity until after installation 
on the ship. Two technologies remain at a lower 
level of maturity— the volume search radar (one of 
two radars that constitute the dual band radar 
system) and total ship computing environment. 
Land-based tests of the volume search radar 
prototype originally planned for before ship 
construction will not be completed until June 
2009 — over 2 years later. Software development 
for the total ship computing environment has 
proved challenging; the Navy certified the most 
recent software release before it met about half of 
its requirements. The Navy plans on completing 89 
percent of product modeling of the ship’s design 
prior to the start of construction. 


Attainment of Product Knowledge 
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Common Name: DDG 1000 


DDG 1000 Program 

Technology Maturity 

Four of DDG lOOO's 12 critical technologies are fully 
mature. Six others are approaching maturity. 
Practical limitations prevent the Navy from fully 
demonstrating all critical technologies prior to 
installation. The Navy does expect to demonstrate 
the maturity of the integrated deckhouse prior to the 
start of ship construction. The Navy conducted the 
deckhouse production readiness review in October 
2008; with completion of a large-scale deckhouse 
test unit in November 2008. Testing of other 
technologies continues through ship construction 
start. The integrated power system will not be tested 
with the control system until 2011 — nearly 3 years 
later than planned, The Navy wiU buy a power 
system intended for the third ship and use it in land- 
based tests. As a result, the power system will not be 
demonstrated until a year after production and 
installation on the two lead ships. 

The volume search radar remains at a lower level of 
maturity. Land-based tests of the volume search 
radar prototype will not be completed until June 
2009 — over 2 years later than planned. Upcoming 
land-based tests will be conducted at a lower volt^e 
than needed to meet requirements — and without the 
radome. The Navy will not demonstrate a fully 
capable radar at its required power output until 
testing of the first production unit in 20 1 1 . Partly due 
to delays, the volume search radar will not be 
installed during deckhouse construction as initially 
planned. Instead, installation will occur in April 
2013 — after the Navy has taken custody of the ship. 

The Navy initially planned to develop and 
demonstrate all software functionality of the total 
ship computing environment (phased over six 
releases and one spiral) over 1 year before ship light- 
off. As a result of changes in the software 
development schedule, the Navy eliminated this 
margin. Until recently, the Navy was able to keep 
pace with its development schedule. However, the 
contractor delivered release 4 without incorporatir^ 
all software system requirements and deferred work 
to release 5, primarily due to issues with the 
command and control component. Problems 
discovered in this release, coupled with the deferred 
work, may be a sign of larger issues that could 


disrupt the development of later releases and 
prevent the timely delivery of software to meet the 
ship’s schedule. 

Design Stability 

The Na \7 aims to complete 89 percent of product 
modeling for the ship’s 94 design zones prior to the 
start of construction. At the program’s production 
readiness reviews in October 2008, the shipbuilders 
had completed less than 35 percent of the product 
model and faced challenges maintaining its design 
schedule. The Navy has now delayed the start of 
ship construction by 4 months to February 2009 in 
order to mature the ship’s design. According to the 
Navy, as of January 2009, 88 percent of the zones are 
complete. 

Other Program Issues 

The Navy recently decided to reduce its quantities 
from seven ships to a total of three. Rather than 
DDG 1000, the Navy now wants to restart the 
procurement of the Aileigh Burke-class destroyer. 
According to the Navy, this is primarily because of a 
change in its assessment of likely future threats and 
in the requirements for destroyers needed to meet 
those threats. While eliminating follow-on ships will 
reduce program procurement costs by at least $10.4 
billion, the costs of the three ships will likely 
increase. Further, the Navy still intends to spend 
$1.6 billion to complete research and development 
of DDG lOOO’s critical systems. 

Program Office Comments 

The Navy stated that the program successfully 
completed production readiness reviews in October 
2008 and that ^most 90 percent of the final Navy 
zone design reviews have been completed, 
emphasizing that no zone will start construction 
until the design for that zone is done. According to 
the Navy, DDG 1000 has a design that is much more 
complete, developed to a greater level of detail, and 
has undergone a more rigorous review than any 
previous ship class. Due to the long timeline 
required to design, develop, and deliver a Navy ship, 
the Navy stated that some concurrency is 
unavoidable to prevent obsolescence and preclude 
the additional cost that would be associated with 
stretching the timeline to allow all technologies to 
reach readiness levels meeting GAO best practice 
recommendations prior to construction. The Navy 
concluded that DDG-1000 has achieved the proper 
balance of developmental risk, schedule impact, and 
cost 
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Common Name: LCS 

Littoral Combat Ship (LCS) 

The Navy’s LCS is (designed to perform mine 
countermeasures, anti-submarine warfare, and 
surface warfare missions. It consists of the ship 
itself — ^the seaframe — and the mission package it 
deploys. The Navy plans to construct the first seven 
LCS seaframes in two unique designs. The first 
seaframe (LCS 1) was delivered in September 2008, 
and the Navy expects the second seaframe (LCS 2) 
to be delivered by September 2009. We assessed the 
first two seaframes (known as Flight 0). See pages 
107-108 for an assessment of LCS mission packages. 


Source; Alien Sctance. 
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Common Name; LCS 


LCS Program 

Technology Maturity 

Fifteen of 19 critical technologies for the two 
seaframe designs are fully mature, and 2 
technologies are approaching maturity. The 
overhead launch and retrieval system in the LCS 1 
design and the aluminum structure in the LCS 2 
design are immature. The Navy identified the 
watercraft launch and recovery concept as a major 
risk to both seaframe designs. This capability is 
essential to complete the LCS anti-submarine 
warfare and mine countermeasures missions. 
According to the Navy, industry watercraft launch 
and recovery designs are unproven. To mitigate risk, 
the Navy is conducting launch and recovery 
modeling and simulation, model basin testing, and 
experimentation and is encouraging the seaframe 
industry teams to adopt similar approaches. Pinal 
integration of mission package vehicles with each 
seaframe will not occur until post-delivery test and 
trials — ^planned first for LCS 1 in 2010 using the mine 
countermeasures mission package, Any problems 
detected could require redesign and costly rework, 
which could delay the introduction of LCS to the 
fleet. 

Design and Production Maturity 

The Navy assesses LCS design stability by 
monitoring changes to requirements documents, 
execution of engineering change proposals, and the 
completion of contract deliverables related to 
drawings, ship specifications, and independent 
certification of the design. Construction is 
monitored using earned value management and 
through evaluation of manufacturing hours spent on 
rework, deficiencies detected and corrected, and the 
number of test procedures performed. 

The Navy adopted a concurrent design-build 
strategy for the first two LCS seaframes, which has 
proven unsuccessful. Contributing challenges 
included the implementation of new design 
guidelines, delays in major equipment deliveries, and 
an unwavering focus on achieving schedule and 
performance goals. These events drove low levels of 
outfitting, out-of-sequence work, and rework — all of 
which increased construction costs. Also, 
incomplete designs during construction led to 
weight increases for both seaframes. According to 
the Navy, this weight growth contributed to a higher 
than desired center of gravity on LCS 1 that 


degraded the stability of the seafirame. In fact, an 
inclining experiment performed during acceptance 
trials showed LCS 1 may not meet Navy stability 
requirements for the damaged ship condition. The 
Navy is taking steps to remove weight and 
implement stability improvements for LCS 1, while 
also incorporating design changes for future 
seaframes. 

Other Program Issues 

As part of LCS 1 acceptance trials, the Navy’s Board 
of Inspection and Survey (INSURV) identified 21 
critical “starred” deficiencies and recommended the 
Chief of Naval Operations authorize delivery of LCS 
1 after correction or waiver of these deficiencies. 
According to Navy officials, only 9 of these 
deficiencies were corrected prior to delivery. Navy 
officials report that transiting the ship away from 
Marinette, Wisconsin, prior to the winter freeze was 
a higher priority than timely correction of starred 
deficiencies. The Navy intends to correct remaining 
deficiencies during planned post-delivery 
maintenance availabilities. The Navy plans to hold 
an INSURV review of LCS 2 upon completion of 
construction and builder’s trials for that seaframe. 

Navy officials report that the earned value 
management systems in each of the LCS shipyards 
do not meet Defense Contract Management Agency 
requirements for validation. Thus, the cost and 
schedule data reported by the prime contractors 
cannot be considered fully reliable by the Navy 
when evaluating contractor cost proposals or 
negotiating for construction of follow-on ships. 

Program Office Comments 

The Navy stated the LCS program is delivering vita! 
capabilities to the fleet and will be a critical 
component of the Navy. It noted that LCS 1 was 
delivered September 18, 2008 — 6 years and 1 day 
after the LCS program was established. In fiscal year 
2009, the program will deliver a second ship of a 
completely different design. According to the Navy, 
while the initial cost and schedule objectives were 
overaggressive — and necessitated a concurrent 
design and construction plan — they provided the 
tension and urgency for these achievements, and 
lessons learned will be applied to future shipbuilding 
programs. In August 2008, INSURV evaluated LCS 1 
and found it to be “capable, well-built, and 
inspection-ready.” The Navy stated it is leveraging 
lessons learned from LCS 1 and LCS 2 to ensure 
future ship awards provide the right mix of 
cap^ility and affordability. 


Page 106 


GAO-09-326SP Assessments of Maior Weapon Programs 



78 


Common Name: LCS Modules 

Littoral Combat Ship - Mission Modules 


The Navy’s Littoral Combat Ship (LCS) will perform 
mine countermeasures, surface warfare and 
antisubmarine warfare missions using modular 
mission packages. Packages include weapons and 
sensors that operate from MH-60 helicopters or 
unmanned underwater, aerial, or surface vehicles. 

Initial packages are partially capable. They include 
engineering development models and some, but not 
all, systems planned. Mission capability improves 
with each package delivered until it reaches a 
baseline capability of production representative 
systems. 

Source: © Norlrtfop Grumman Corporation. 
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Operation of the Mine Countermeasures (MCM), 
Surface Warfare (SUW), and Antisubmarine 
Warfare (ASW) packages requires a total of 25 
critical technologies, including 13 sensors, 5 
weapons, and 7 vehicles. Technology 
development has proceeded more slowly than 
expected. Individual mission systems in each 
package have experienced problems requiring 
design changes and resulting in schedule delay. 
For example, integration of MCM systems with the 
MH'60S helicopter has proved challenging due to 
problems with the cable that tows the various 
systems. Overall, the Navy will reach baseline 
capability for each package between 1 and 2 years 
later than previously planned. The Navy plans to 
procure 64 mission packages for use on 55 LCS 
seaframes. Procurement has slowed to keep pace 
with seaframe acquisition. 


Attainment of Product Knowledge 



(NA) 
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Common Name: LCS Modules 

LCS Modules Program 

Technology Maturity 

Operation of the MCM, SUW, and ASW packages on 
the LCS requires a total of 25 critical technologies, 
including 13 sensors, 5 weapons, and 7 vehicles. Of 
these technologies, 17 are currently mature and 8 
are nearing maturity. 

The first of 24 MCM packages was delivered in 
September 2007 and included 7 of 10 planned 
mission systems. Four systems are not yet mature; 
two of these are struggling to reach full maturity. 
Officials note the Organic Airborne and Surface 
Influence Sweep is being redesigned to address 
corrosion issues and the Rapid Airborne Mine 
Clearance System requires design changes to 
perform in all environmental conditions. An 
airborne mine countermeasures system was 
decertified and its tow cable is being redesigned 
following the results of testing with the helicopter. 
The Navy also decertified the Remote Minehunting 
System during testing in 2007 due to reliability 
issues, and, according to officials, results of a recent 
operational assessment are pending. The Navy now 
plans to deliver the third and fourth mission 
packages in fiscal year 2011 and has delayed delivery 
of the baseline package until fiscal year 2012. 

The first of 24 SUW packages was delivered in July 
2008 and included 1 of 2 planned mission systems. 
The SUW package includes the fully mature 30mm 
gun and a variant of the Array’s Non-Line-of-Sight 
(NLOS) system (missile and launcher), which is 
nearing maturity. The first package consisted of two 
gun engineering development models, without the 
NLOS launcher or missiles. The NLOS design for 
LCS has not yet been validated. Integration of the 
gun with LCS is not complete. A design review for 
the gim module is scheduled for October 2009. 
Delivery of a baseline package has been delayed to 
fiscal year 2013. 

The first of 16 ASW packages was delivered in 
September 2008 and included 4 of 10 planned 
mission systems. Three systems remain immature 
Including the Unmanned Surface Vehicle’s Dipping 
Sonar, the Remotely Towed Array and the Remotely 
Towed Array Source. Failure to develop these 
technologies as expected could increase reliance on 
the MH-60R helicopter. The Navy has delayed 


delivery of a second ASW package until fiscal year 
201 1, and delayed basehne capability from fiscal 
year 2011 to 2013. 

Other Program Issues 

The development cost of the LCS packages has 
increased by more than $300 million, or 64 percent 
since last year. Procurement costs have decreased 
for MCM, in part because the delivery of the more 
expensive baseline capability has been delayed. 
Reductions in fiscal year 2008 and 2009 budget 
requests have slowed mission package procurement 
to account for continuing delays in seaframe 
acquisition. The explanatory statement 
accompanying DOD Appropriation Act for Fiscal 
Year 2009 Congress asked the Navy to develop a plan 
for fielding the MCM capability independent of LCS. 
The program office indicates all packages are 
currently scheduled to undergo operational 
assessments with both LCS seaframe designs, 
beginning in June 2010. According to program 
officials, in September 2008, the Navy conducted a 
shore based integration exercise using simulated 
seaframe mission bays. Officials note this activity 
accelerated MCM mission package integration with 
both seaframes and reinforced previous crew 
training. 

Program Office Comments 

Program officials noted that changes to the program 
between the 2008 and 2009 president’s budgets 
resulted in an apparent increased development cost. 
Costs for the SUW package bought in fiscal year 
2009 were realigned from procurement to 
development to support technical and operational 
evaluations. In addition, data provided to GAO for 
last year’s assessment did not include costs of 
common equipment that was subsequently 
distributed among the MCM and ASW packages. The 
program office acknowledges technical maturity 
challenges for some mission systems and is working 
closely with mission system program offices to 
resolve any issues. The program office is leading a 
coordinated test approach to prove mission package 
capabilities and suitability for fleet delivery. The 
program office also provided technical comments 
that were incorporated as appropriate. 
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Senator Martinez. My time’s up, thank you very much. 

Chairman Levin. Thank you, Senator Martinez. 

Senator Begich. 

Senator Begich. Thank you. Thank you very much, Mr. Chair- 
man. 

It has been very interesting, listening to all the testimony. I will 
try to ask my questions very quickly, and I may not ask every sin- 
gle one of you to answer because I want to get some specifics here. 
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Before I talk about the personnel issue, I’m a former mayor; I’ve 
just become a Senator. So, I believe I’m a mayor who just happens 
to be a Senator, so I like some of the comments you’ve made on 
personnel issues that I’ll talk about in a second. 

I’m not familiar with the CAIG. Is that the CAIG? I looked 
through the bill as quickly as I could, and I see an annual report 
to Congress. Do you think it would be helpful for us to require 
more? I believe it’s a combination of oversight; it’s not just internal 
oversight, but this body needs to do more. I think this bill attempts 
to do that. But, do you think there should be a report — and maybe 
there is and I’m just not familiar with it — that comes to this com- 
mittee on a regular basis, maybe quarterly or twice a year, that 
shows what the CAIG said it would cost before a system is started? 

Mr. Gansler, you made the comment that — I may get these words 
wrong — ^but it’s almost like they ignore it. It comes out, it says, 
maybe, what it might really cost, but then they kind of push it 
down. I’m familiar with this, as a mayor. We call them HMS, Inc., 
studies, where the contractor wants to build it, as well the person 
inside the system who wants to build it, always seems to have a 
different price. When it’s all done, it’s pretty close to the one that 
HMS, Inc., did. Is that something we should have in this legisla- 
tion, so we can see, before these systems start kicking off, here is 
what the real, or what another group said it could cost, so we at 
least have some understanding? I don’t know who can answer that. 

Dr. Gansler. The idea of the independent cost analysis, right up- 
front, when we’re doing the early systems engineering, is to be able 
to see the impact of the various requirements, and to be able to 
trade requirements, schedule, and cost as part of that early design 
requirements setting. Cost is actually an engineering challenge, 
just as schedule and technical performance are. So, if you, upfront, 
try to say, “What’s the cost impact of this?” and the independent 
cost analysis group will say, “Historically here’s what it’s been,” 
and then people will come back and say, “But, this time it’s going 
to be different.” 

Senator Begich. Everything’s different the next time. 

Dr. Gansler. Right, and so it’s important to keep that in per- 
spective, that the new technology comes along, and so forth. But, 
it’s absolutely essential to get that independent estimate. Now, do 
you want to have Congress legislate what the price should be? I 
don’t think so. 

Senator Begich. No, I’m not asking that. But here we are, com- 
plaining about all these cost overruns. I agree with all your com- 
ments. You can write all the legislation you want, but you do not 
change the culture and remove people who are not doing the job 
they should be doing, and putting people in there who should be 
doing the job, you don’t change anything, we’ll be back here in a 
couple of years. So, I’m not saying, legislate the price, but we be- 
come more aware, so we are putting the pressure where it should 
be. Let’s be honest about the pricing, so when we do the budgeting 
authorization and appropriation, we don’t go from 75 to 95, with 
two-thirds probably in the planning and design stage, and we al- 
ways get the answer, “We’re this far, we have to do a little bit 
more, a little bit more, a little bit more.” So what I’m asking is, 
should we have a more regular reporting period? Because once 
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you’re a year into a project, even though some are longer, they’re 
already obligating more money. By the time this system here 
moves, you could be 2 years into having any commentary on it. 

That’s my question. Should it be more regular reporting? 

Dr. Gansler. I think the thing you’re missing is the fact that, 
in the development of a program, the program changes weekly. 
There’s always technical issues that come up, there’s always sched- 
ule problems, there’s personnel problems, and so forth. You don’t 
want Congress to be micromanaging the programs. You do want to 
make sure that the process is a good process. That’s where I think 
the independent cost analysis is a very important thing and that 
your emphasis on it is the right thing to do. But, I don’t think you 
want to get down to the point where you have a weekly report from 
the DOD. 

Senator Begich. I didn’t ask that. I’m trying to get to the point. 
Is annual enough? 

Dr. Gansler. I think that — if you’ve convinced the DOD that you 
care about this, that you are going to be monitoring it, and that’s 
what you’re trying to do with the threshold numbers and the con- 
trols on that — I think that you’re giving the message that the DOD 
needs to care about cost. 

Senator Begich. Okay. Let me make one comment. I forget who 
said it, about, “stop the obligation of money at a certain point.” I 
know, as a mayor, that’s what you do. You turn the dollars off, and 
suddenly you get a response, and you get people paying more atten- 
tion. So, I might be a little different than earlier comments that 
were made. I’m a little more direct on that. But, the personnel 
issue, to me, this is the challenge. If you don’t change the culture, 
nothing changes. We’ll be back here, and the numbers will grow, 
as they have over the years. It’s not about adding more people, and 
I think Congress made a huge error by reducing down the amount 
of people. That was a huge mistake. We basically took the people 
who manage our programs out of the equation. 

So, besides putting more people into the system to make sure we 
have more folks out there, do you think — and I’m not sure I want 
to ask this question because I’m not sure you’ll want to answer it — 
but, do you think, within the system that currently exists today, we 
have to change the deck? When I say “change the deck,” change 
personnel, people, and not just add more people and move people 
around so we satisfy their issues. This is a very hard, direct ques- 
tion. I’ve had to do this, as mayor. You might have half a dozen, 
a dozen, or 100 people; it doesn’t really matter — they’re in the 
wrong place. Anyone dare to want to answer that question? 

Mr. Sullivan. I would just say, just real quickly, that a lot of 
this is organizational. I think the people that work in the DOD now 
are great people, and really, really capable people, and good public 
servants and everything else. 

Senator Begich. Right. 

Mr. Sullivan. But, when we talk about culture, I guess, it’s 
more the way things are organized. For example, I think, in our 
written statement, we have that the Under Secretary of Defense for 
Acquisition, Technology, and Logistics probably should have more 
ability to make the final decisions on things, acquisition, than he 
or she has now. One of the things that probably gets in the way 
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of that is the fact that that position had an average turnover of 20 
months since it started in 1986. I think that’s part of the problem; 
there’s an accountability issue, people change over too much, 
there’s not a lot of direct communication. The three processes, I 
think, that this legislation does address, that you’re trying to get 
the three big processes that we’ve talked about to communicate 
with each other more and to share in decisionmaking; right now, 
that’s not there. So, I don’t know if the people are good people. The 
structures and the way they’re organized and the way they come 
and go is the big issue. 

Dr. Gansler. If I might comment that if we implemented what 
Goldwater-Nichols says to do, relative to promotion potential for 
the acquisition community, that you’d get a big step forward there. 
Instead of putting someone into a four-star position who has no ac- 
quisition background, but happens to be called an acquisition job, 
that’s where we lose out. Each of the promotion-potential reviews 
and so forth need to really show that we value the acquisition 
workforce, civilians and military, and that’s a critical point, I think, 
in order to keep people coming in. Dr. Kaminski said they’re not 
doing it for the money, they’re doing it so they can have an impact, 
and they need to have promotion potential. 

Senator Begich. I know my time’s out, but you’re about to jump 
out of your seat, Mr. Adolph. 

Chairman Levin. Can you do it real quickly? 

Mr. Adolph. Very quickly. There are three issues: numbers, 
training, and people. Particularly in the Air Force, they need to 
plus up because they drew down their acquisition workforce to a 
greater extent. 

I think the training that Defense Acquisition University provides 
is on the mark, for the most part. They were a part of our study. 
They ground in what we found into their training courses. I believe 
people need to be moved around a bit more, particularly in the ci- 
vilian workforce, so that they get a variety of experiences rather 
than 1 or 2 years’ experience 10 times. 

Senator Begich. Thank you very much, Mr. Chairman. 

Chairman Levin. Thank you. Senator Begich. 

Senator Lieberman. 

Senator Lieberman. Thanks, Mr. Chairman. Thanks to the 
panel. It’s probably not been an exciting hearing in content, but it’s 
actually very important, an educational hearing, and there’s a lot 
on the line in it, so I thank the four of you for the accumulated 
experience and wisdom you’ve brought to the table. I thank Senator 
Levin and Senator McCain for their legislation, S. 454, which cer- 
tainly would take us forward in significant ways. I hope it passes. 

Earlier, Senator Collins referred to the fiscal responsibility sum- 
mit that President Obama convened last week at the White House, 
and then a group of us on this committee happened to be in a 
breakout session on procurement reform. As Senator Collins indi- 
cated, most, though not all, but most, of us focused on the acquisi- 
tion workforce, and the size of it. I want to talk to you about that 
in the time I have, hopefully, and at least one other subject we 
talked about. 

Dr. Gansler, you had a chart here in your testimony which has 
two lines; one is procurement by DOD, in dollars, from 1990 to 
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2006, about as close as we can get to today, and then the other is 
the acquisition workforce. Obviously, the procurement dollars go up 
dramatically as the acquisition workforce goes down. But, beneath 
that, there are some numbers that are quite stunning, I think, and 
I know when I was here, Mr. Sullivan referred to them. The acqui- 
sition workforce, in 1990, was actually 500,000 people, and today 
it’s dropped, but it’s still 200,000 people, which is an enormous 
number of people in acquisition. I note in your testimony that you 
focus in on the DCMA and say that it had 25,000 people in 1990, 
down to 10,000 today. Then the other four general officers, and 
down to zero today. Give me some sense of the 200,000. Because 
my first reaction to it is, “Wow, 200,000 people, isn’t it enough to 
handle acquisitions by the DOD, even though acquisitions are so 
large?” 

Dr. Gansler. a large share of those are in the military depots 
that you have insisted do 50 percent of the maintenance work. A 
depot that has 20,000 people, that adds up pretty fast. To get to 
200,000, you only need 10 of those. 

Senator Lieberman. So, acquisition wouldn’t be what most of 
those do? 

Dr. Gansler. That’s part of the acquisition workforce because lo- 
gistics is part of the acquisition. 

Senator Lieberman. Got it. But in the conventional meaning, I 
think that’s important, so I wanted to bring out. 

Dr. Gansler. Very few people are actually doing contract work, 
program management work, or things like that. As Mr. Adolph 
pointed out, the test and evaluation community is down signifi- 
cantly, but they’re part of that community. So it’s the total encom- 
passing the research labs that the government runs all the way 
through the maintenance and logistics support. 

Senator Lieberman. Is there any way, now or maybe afterward, 
to submit for the record, you could give us a sense of what we 
would normally, in conversation, consider to be the acquisition 
workforce — how many people in the DOD are actually involved in 
acquisition, contracting, et cetera? 

Dr. Gansler. It’s a small percentage of the people that are actu- 
ally involved in that. 

In other words, you have the comptroller people, you have the 
personnel people, you have the policy people, and you have the ac- 
quisition workforce. 

Senator Lieberman. Right. I’m curious as to really what the size 
of the real acquisition workforce is, leaving out the depots and the 
rest. 

Mr. Sullivan. I don’t have that answer right now. I know we’re 
doing work on that. 

Senator Lieberman. Good. Get it for the record, please. 

Mr. Sullivan. Yes, sir. 

[The information referred to follows:] 

In our recent review of the Department of Defense’s (DOD) acquisition workforce 
management (GAO-09-342), we found that, according to DOD data, the number of 
civilian and military personnel in the Department’s acquisition workforce totaled 
about 126,000 at the end of fiscal year 2007 compared to about 129,000 personnel 
in 2001, a decline of about 2.5 percent. During this same time period, the number 
of contracting actions valued at over $100,000 increased by 62 percent and dollars 
obligated on contracts increased by 116 percent, according to DOD. Moreover, DOD 
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has reported that the number of major defense acquisition programs has increased 
from 70 to 95. To augment its declining in-house acquisition workforce, DOD has 
relied more heavily on contractor personnel. However, DOD does not collect or track 
information on contractor personnel, despite their being a key segment of the total 
acquisition workforce. 

Senator Lieberman. There were two other things that there 
seemed to he an interesting consensus on, and they’re quite dif- 
ferent, about, you might say, principles for acquisition; a little dif- 
ferent than anything we’ve talked about today. They’re not unfa- 
miliar. One was that our original position on acquisition should be 
to favor fixed-price contracts over cost-plus, and to favor competi- 
tive bidding, as opposed to negotiated contracts. I want to ask, to 
the extent that we have time — let’s just focus on the fixed-price. 
Our sense, as we discussed at our breakout session with probably 
about 25 people, was, generally speaking, private sector favors com- 
petition. So, why are we favoring cost-plus? Does the taxpayer real- 
ly benefit from that? 

Mr. Adolph, you always get asked last because we’re going left 
to right, so let’s start from the right and ask you about that. 

Mr. Adolph. I think the other three panel members have more 
expertise in this issue. But, when you’re in the basic research 
arena, it’s very difficult to do on a fixed-price basis. It should be 
accomplished using cost-plus contracts. 

Once you get beyond research, then the next challenge in the de- 
velopment is system integration. That’s not an insignificant task 
with the very complex systems we’re developing today. But, once 
you get beyond that point then I think you reach a point where you 
can really consider going to fixed-price for downstream procure- 
ment. 

Senator Lieberman. Right. That’s interesting. 

Dr. Kaminski, I think, just to clarify — if I could really make it 
too simple — what if we had a law that said, “Defense contracts 
ought to be done on a fixed-price basis unless the Secretary cer- 
tifies that there’s a good reason not to”? 

Dr. Kaminski. I think that would end up requiring a lot of certifi- 
cation, Senator Lieberman, for the following reason. 

Senator Lieberman. Right. 

Dr. Kaminski. I believe fixed-price contracts are completely ap- 
propriate when we know precisely what it is we’re going to buy. If 
there’s uncertainty in what we’re going to buy, and we know we’re 
going to change, and we don’t know yet quite how we’re going to 
change, I think we end up on the wrong end of the bargain negoti- 
ating a fixed-price contract and then having to go back and renego- 
tiate that effort for every change that occurs, especially when the 
contractor has already priced in some contingency in the fixed 
price. 

So, there’s a time, for example, in the program, where perhaps 
we are working through this in development, and then we settle in 
on what we want to buy, and we’re ready to enter a well-defined 
production program. That would be a fine time to do a fixed-price 
contract. 

So, I think you have to pass that criteria, knowing what it is you 
want to buy, and have it be predictable. 
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Senator Lieberman. Okay. My time is running out, but, Mr. Sul- 
livan, Dr. Gansler, if you’d give me a quick answer to a big ques- 
tion and follow up with writing. 

Mr. Sullivan. For major developments, I think it would be very 
difficult to go to a fixed-price contract for that because of the un- 
knowns that are involved. But, I would say that if you work on re- 
quirements and try to do some of the things that we’ve been talk- 
ing about here today, in terms of staying in what is doable, and 
having shorter cycle times to get these things done, you could have 
cost-plus development contracts that don’t get so out of control. It 
really goes back to how much knowledge you have when you set 
out. 

Senator Lieberman. Okay. 

Dr. Gansler? 

Dr. Gansler. I think that it’s a question of what’s the meaning 
of a “fixed-price contract”? As Senator McCain said, you get 75 
changes every week. The contract continues to change hourly, in ef- 
fect. I think it’s very clear, when you have a stabilized and lower- 
risk program, that a fixed-price makes a lot of sense, it does give 
an incentive for the contractors. On the other hand, the cost-plus, 
I would say, we haven’t been using the incentives that are avail- 
able with the cost-plus-type contracts as well as we should, and I 
think, clearly, for research-and-development-type activities, cost- 
plus is an appropriate way to do it, but the “plus” part is an incen- 
tive rather than a fixed fee, I think. I would use the incentive 
more. 

Senator Lieberman. Those are very helpful answers. You encour- 
age me to think that we ought to take a look, not at fixed prices 
on across-the-board answer, but to apply it by some selective 
means, and that, in doing so, we might benefit the taxpayers. 

Thanks very much. 

Chairman Levin. Thank you. Senator Lieberman. 

Senator Webb. 

Senator Webb. Thank you, Mr. Chairman. I’d like to add my ap- 
preciation for your taking on this issue. As has been so clear today, 
we’re burning up a lot of money, and we’re not getting a lot of prod- 
uct right now, particularly in the shipbuilding programs and air- 
craft programs. 

Dr. Gansler, when you’re talking about people, which everyone 
seems to agree is the major issue, I was thinking about all the dif- 
ferent years and different positions I’ve had on different sides of 
the table, here, working on this issue. It’s so clear that what we 
need is disciplined management, not only on the people side, but 
in the system itself. “People” include people in government, on this 
side of government, it includes people in business. We have chal- 
lenges because there are not a lot of people in the military who use 
the business concepts, quite frankly, and they’re asked to manage 
these programs. There are not a lot of businesspeople who are used 
to how product comes through a governmental system. 

I believe it was Mr. Sullivan who mentioned the creation of the 
Under Secretary of Defense for Acquisition in 1986. I was actually 
on Caspar Weinberger’s staff when that position was created. We 
had a very talented individual who came into the position. He was 
bewildered with all the different steps that were required to get a 
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system through the process. It’s something you just don’t see in the 
normal business world. There are lots of checks and balances. Some 
of them are appropriate, and some of them are less than appro- 
priate. 

Dr. Gansler, I found your analogy with the LCS brought back a 
lot of different memories. They say that a camel is a horse created 
by a committee. I can go all the way back to the M-16, when they 
first developed the M-16, and they developed it directly toward a 
jungle environment, and then the different requirements were put 
on it to be able to be used in the desert and other different places. 
They put a different round in it that carboned up the chamber so 
people were dying in Vietnam because the weapon system require- 
ments had changed as it evolved. 

I was in DOD when we were trying to do the Bradley fighting 
vehicle, and there were different requirements put on it here in 
Congress, so that it was very similar to what you were talking 
about with the LCS. They were saying it should perform different 
functions from the original design, and then there were all these 
press reports about the Bradley fighting vehicle falling over when 
it was going through a water obstacle because it got top heavy. 

Or the FG-7s, the USS Sfar^-class ships, which were designed 
to build to cost. So, we have fixed cost that we were going to build 
a ship toward, and then you go inside one of these FG-7s, you 
could plink the bulkheads on a FG-7, they were so thin. So when 
an Exocet missile hit the USS Stark in the Persian Gulf when I 
was Secretary of the Navy, it went all the way through the ship 
because they had had to make adjustments based on the cost rath- 
er than on some other areas. 

So it’s a very complicated question. I think the key, when I look 
back, is if you can find the right leadership at the top on a pro- 
gram, negotiate and agree on general requirements — there are al- 
ways going to be fixes — we’ll get a program through. Probably the 
best example of that is when they put A1 Gray, who later became 
Commandant of the Marine Corps, and he had the Development 
Center, they put him on the Light Armored Vehicle, and he got 
that program through in about a year. He just pushed it through, 
made all the negotiations, was very firm with people over here in 
Congress, as to what the requirements were for the Marine Corps, 
and it was a very successful program. 

I have one question I would like to put in front of you because 
I’m very concerned about it, and that is the state of all these pro- 
grams in the United States Navy right now. You talk about the Po- 
laris as having been probably the best analytical prototype of how 
to build a weapon system. One of the things about our submarine 
programs is that we built the frame, and then we added the tech- 
nology onto the frame, similar to, say, the C-130, rather than con- 
tinually building a new frame with all the costs and the time that 
goes into that. I’ve just been really struck over the last couple of 
years at how difficult the Navy procurements appear to be, and I’m 
trying to get my arms around why. I would be happy to hear from 
any of you who would like to begin answering that question. 

Dr. Kaminski. I’ll make one comment. I think a key thing that 
impacts the programs are stability. So, if you see a program that 
we’ve had in production for some period of time, they have very 
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good learning curves on those ships when we have a stable produc- 
tion program. Our issues sometimes are with first-ship cost. But, 
if you look at the cost of subsequent ships, what’s happening there 
is very competitive with commercial-kind of production experience. 
Where we’re producing ships regularly — one of the ship families in 
which we’re doing that is the DDG-51. We have two yards. We 
have some competitive arrangements. Not quite head-to-head com- 
petitions, but there are some incentives in those programs. But, it 
is a well-planned, stable program that we’ve been producing. I 
think that approach would benefit us. It’s the areas where we’ve 
had instability where we’ve more problems. 

Dr. Gansler. Yes, I guess I would approach it by thinking about, 
“What is it the Navy really needs for the 21st century?” and what 
types of ships they’re going to want. There is a resistance to 
change. Take the Arsenal ship, for example, which was primarily 
to support the Marines and the Army onshore, it was resisted sig- 
nificantly in terms of it being a low-cost ship for few people. The 
highest costs in the Navy are for people and fuel if you look at the 
life-cycle costs of a ship. So trying to drive down the number of peo- 
ple on the ship and improve the fuel utilization are things we need 
to stress. Those are not the traditional things that are emphasized 
in the Navy construction of ships. I think it’s a different look that 
we need to think about. The LCS has the same thing. Is that some- 
thing the Navy really wanted or really resisted? So it’s more the 
institutional inertia that has to be changed, I think, in terms of 
what the future Navy for the 21st century is going to need. 

Senator Webb. Do any of you see this as leadership failures in 
the Navy? [No response.] 

In terms of defining these objectives? 

Dr. Gansler. There have been some real successes. The F-18E/ 
F on the Navy program was extremely well-managed, but that was 
because they had some really top people doing it, they had a clear 
objective, it was a incremental version of a prior demonstrated pro- 
gram, and it was well done. 

Another big success is the patrol frigate. It kept up competition 
throughout its life, and it had the steepest learning curves of any 
ship in the Navy. So, there are some success stories, but I think 
lessons learned haven’t been widely applied. 

Senator Webb. Thank you, Mr. Chairman. My time’s up. 

Chairman Levin. Thank you. Senator Webb. 

Any other questions of this panel before we excuse them? [No re- 
sponse.] 

Thank you all for your time. Some of you came some distance to 
get here, at some inconvenience. At least one of you had to give up 
a family commitment, and we won’t identify who that was because 
the family is better off not knowing, maybe. [Laughter.] 

But we’re very grateful to all of you for your testimony. It’s very, 
very helpful. 

I will also submit for the record the text of the Weapon Systems 
Acquisition Reform Act of 2009; a statement from Ken Krieg, 
former Under Secretary of Defense for Acquisition, Technology, and 
Logistics; a statement from Moshe Schwartz from the Congres- 
sional Research Service; and the DOD Inspector General Acquisi- 
tion Workforce Count Report. 
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llTH CONGRESS >f ^ if 

1ST Session 4D4 

To improve the oi'g^anizatioii and pi*()eedures of tiie Department of Defense 
for tlie ae(iuisition of mi\jor weapon systems, and for othta- purposes, 


IN THE SENATE OF THE UNITED STATES 

FEltliUAltY 2:!. 2009 

Mr. Levix (for himself and Mr. McCain) iiitrodueed the foltowiiiff bill; wliich 
was read twiee and referred to the Committee on ilrmed Services 


A BILL 

To improve the organization and procedures of the Depart- 
ment of Defense for the acquisition of major weapon 
systems, and for other pui-poses. 

1 Be it enacted by the, Benate and House of Representa- 

2 fives of the United. States of America in Congress assembled, 

3 SECTION 1. SHORT TITLE; TABLE OF CONTENTS. 

4 (a) Short Title. — This Act maj^ be cited as the 

5 “Weapon Systems Acquisition Reform Act of 2009”. 

6 (b) Table of Contents. — The table of contents for 

7 this Act is as follows: 

See. 1. Shorl; Lille; table of wnitents. 

ISec 2. I’oiion'ssiotuil defense coniniittees. 


'I'n'LR I— A(;QI:I8ITK)\ OKGAXIZATIOX 
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2 

See. 101. Reports on systems enflrineering^ ca[)abilities of the D(‘pai'tnient of De- 
fense. 

Sec^. 102. Dir<!(^tor of Developmental Test and Evaluation. 

See. 100. .^\ssessmont of teehnolofrical maturity of mtieal teehnologies of major 
defen.se a<tipusilion progi'ams by the Direetor of Defense Re- 
.scareh and Rnjfineerinw. 

See. 104. Direetoi- of Independent Cost Assessment. 

See. lOf). Role of the eonnnandei's of the combatant eonnnand.s in identifying: 
joint militaiy requirements. 

TITLE II— ACQUISITION POLICY 

Sec. 201. Consideration of trade-offs among cost, sehctlule, and {)orformanee in 
the ae(|uisition of major weapon systems. 

Se(‘. 202. Preliminary design review for major defense acquisition [nograms. 

See. 203. Maximi:<ation of coin)>etition throughout the life eyele of !iia.jor de- 
fense ae.(|nisition piDg^’ams. 

See. 204. Critical cost gimth in major defense awiuisition [)n)gi*ams. 

Sec. 205. Organizational eonllicts of interest in the acquisition of major \vea[)on 
•systems. 

See, 206. Awards for Department of Defense pers<mnel for excellence in the hc - 
quisition of piwlucts and semccs. 

1 SEC. 2. CONGRESSIONAL DEFENSE COMMITTEES. 

2 III this Aet, the term “eoiigressioiial defense commit- 

3 tees” has ttie meaning given that term in section 

4 101(a) (16) of title 10, United States Code, 

5 TITLE I— ACQUISITION 

6 ORGANIZATION 

7 SEC. 101. REPORTS ON SYSTEMS ENGINEERING CAPABILI- 

8 TIES OF THE DEPARTMENT OF DEFENSE. 

9 (a) Reports by Servtce Alqi'isition' Execu- 

10 TiVES. — Not later than 180 daj's aftei' the date of the eii- 

11 aetment of this Act, the sei-vice acquisition executive of 

12 each militaiy department shall submit to the Under Scc- 

13 retarv' of Defense for Acquisition, Technology, and Ijogis- 

14 tics a report setting forth the following: 
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1 (1) A description of the extent to which such 

2 niihtary department has in place development plan- 

3 ning' organizations and processes staffed by adequate 

4 numbers of personnel with appropriate training and 

5 exi>ertise to ensure that — 

6 (A) key requirements, acquisition, and 

7 budget decisions made for each major weapon 

8 system prior to Milestones A and B are sup- 

9 ported by a rigorous systems analysis and sys- 

10 terns engineering' process; 

11 (B) the systems engineering strategy for 

12 each major weapon system includes a robust 

13 program for improraig reliability, availability, 

14 and maintainability as an integral part of de- 

15 sign and development; and 

16 (C) systems engineering requirements, in- 

17 eluding reliability, availability, and maintain- 

18 ability requirements, are identified during the 

19 Joint Capabilities Integration Development Sys- 

20 tern process and incorporated into contract re- 

21 quirements for each major w'eapon system. 

22 (2) A description of the actions that such mili- 

23 taiy department has taken, or plans to take, to — 
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1 (A) establish needed development planning 

2 and systems engineering organizations and 

3 processes; and 

4 (B) attract, develop, retahi, and reward 

5 systems engineers with appropriate levels of 

6 hands-on exjjerienee and technical expertise to 

7 meet the needs of such military department. 

8 (b) Report by Under Secretary of Defense 

9 FOR ACQL'ISITION, TECHNOLOGY, AND LOGISTICS. — Not 

10 later than 270 days after the date of the enactment of 

1 1 this Act, the Under Secretary of Defense foi’ Acquisition, 

12 TechnologJ^ and Logistics shall submit to the Committee 

13 on Armed Services of the Senate and the Committee on 

14 Armed Services of the House of Representatives a report 

15 on the system engineering capabilities of the Department 

16 of Defense. The report shall include, at a minimum, the 

17 following: 

18 (1) An assessment by the Under Secretary of 

19 the reports submitted by the service acquisition ex- 

20 ecutives pursuant to subsection (a) and of the ade- 

21 quacy of the actions that each military department 

22 has taken, or plans to take, to meet the systems en- 

23 gineering and development planning needs of such 

24 military department. 
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1 (2) All assessment of each of the recommenda- 

2 tioiis of the report on Pre-Milestone A and Early- 

3 Phase Systems Engineering of the Air Force Studies 

4 Board of the National Research Council, including' 

5 the recommended checklist of systems engineering 

6 issues to be addressed jirior to Milestones A and B, 

7 and the extent to which such recommendations 

8 should be implemented throughout the Department 

9 of Defense. 

1 0 SEC. 102. DIRECTOR OP DEVELOPMENTAL TEST AND EVAL- 

1 1 UATION. 

12 (a) Establishment op Position. — 

13 (1) In general. — Chapter 4 of title 10, United 

14 States Code, is amended by inserting after section 

15 139b the following new section: 

16 “§139c. Director of Developmental Test and Evalua- 

17 tion 

18 “(a) There is a Director of Developmental Test and 

19 Evaluation, who shall be apfiointed by the Secretaiy of De- 

20 fense from among individuals with an expertise in acquisi- 

2 1 tion and te.sting. 

22 “(b)(1) The Director of Developmental Test and 

23 Evaluation .shall be the principal adidsor to the Secretary 

24 of Defense and the Under Secretaiy of Defense for Acqui- 
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1 sitioii, Technology, and Logistics on developmental test 

2 and evaluation in the Department of Defense. 

3 “(2) The Director shall be subject to the supervnsion 

4 of the Under Seeretan’ of Defense for Acquisition, Tech- 

5 uology, and Logistics and shall report to the Under Sec- 

6 retarje 

7 “(c) The Director of Developmental Test and Evalua- 

8 tion shall — 

9 “(1) develop policies and guidance tor the devel- 

10 opmental test and evaluation actmties of the De- 

1 1 partment of Defense; 

12 “(2) monitor and roUew the developmental test 

13 and evaluation activities of the Department of De- 

14 fense; 

15 “(3) review and approve; the test and evaluation 

16 master plan for each major defense acquisition pro- 

17 gram of the Department of Defense; 

18 “(4) supervise the activities of the Director of 

19 the Department of Defense Test Resource Manage- 

20 ment Center under section 196 of this title; 

21 “(5) review the orgatiizations and capabilities of 

22 the military departments with respect to develop- 

23 minital test and evaluation and identify needed 

24 changes or improvements to such organizations and 

25 capabilities; and 
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1 “(6) perform such other activities relating to 

2 the developmental test and evaluation activities of 

3 the Department of Defense as the Under Secretary 

4 of Defense for Acquisition, Technology, and Logis- 

5 tics may prescribe. 

6 “(d) The Director of Developmental Test and Eval- 

7 nation shall have access to all records and data of the De- 

8 partment of Defense (including the records and data of 

9 each military department) that the Director considers nec- 

10 essary in order to carrj' out the Director’s duties under 

1 1 this section. 

12 “(e) The Dii-eetor of Developmerrtal Test arrd Evalua- 

13 tiorr shall submit to Corrgress each year a report on the 

14 developmental test and evaluation activities of the Depart- 

1 5 merrt of Defeirse dur-ing the precedirrg year. ’ ’ . 

16 (2) Clerical amendment. — T he table of see- 

17 tions at the beginning of chapter 4 of such title is 

18 amended by inserting after the item relating to sec- 

19 tion 139b the following new item; 

Director of DevelojmiCDtal Test and Evaluation.’". 

20 (3) CONEOKMLNC AMENDMENT. — Section 196(f) 

21 of such title is amended by striking “the Under Sec- 

22 retary of Defense for Acquisition, Technology, and 

23 Logistics” and all that follows and inserting “the 

24 Under Secretary of Defeirse for Aisriuisition, Tech- 
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1 iiology, and Logistics and the Director of Develop- 

2 mental Test and Evaluation.”. 

3 (b) Reports on DEtTiLOPMENTAL Testing Okga- 

4 NIZATIONS AND PERSONNEL. 

5 (1) Reports by servtce acquisition enecu- 

6 tives. — Not later than 180 days after the date of 

7 the enactment of this Act, the seiviee acquisition ex- 

8 ecutive of each militarj' department shall submit to 

9 the Director of Developmental Test and Evaluation 

10 a report on the extent to which the test orgaiiiza- 

11 tions of such military department have in place, or 

12 have effective plans to develop, adequate numbers of 

13 personnel with appropriate expertise for each pur- 

14 pose as follow’S; 

15 (A) To ensure that testing requirements 

16 are appropriately addressed in the translation 

17 of operational requirements into contract speci- 

18 fieations, in the source selection process, and in 

19 the preparation of requests for proposals on all 

20 major defense acquisition programs. 

21 (B) To participate in the planning of de- 

22 velopmental test and evaluation aetmti(;s, in- 

23 eluding the preparation and ai)proval of a test 

24 and evaluation master plan for each major de- 

25 fense acquisition program. 
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1 (C) To participate in and oversee the eon- 

2 duct of developmental testing, the analysis of 

3 data, and the preparation of evaluations and re- 

4 ports based on such testing. 

5 (2) First ^\nnual kerokt by director or 

6 DEVELOFMENTiUj TEST AND EVALUATION. — The 

7 first annual report submitted to Congress by the Di- 

8 rector of Developmental Test and Evaluation under- 

9 section 139c(e) of title 10, United States Code (as 

10 added by subsection (a)), shall be submitted not 

1 1 later than one year after the date of the enactment 

12 of this Act, and shall include an assessment by the 

13 Director of the reports .submitted by the service ac- 

14 (juisition executives to the Director under paragi-aph 

15 (1). 

16 SEC. 103. ASSESSMENT OF TECHNOLOGICAL MATIHUTY OP 

17 CRITICAL TECHNOLOGIES OF MAJOR DE- 
IS FENSE ACQUISITION PROGRAMS BY THE DI- 

19 RECTOR OF DEFENSE RESEARCH AND ENGI- 

20 NEERING. 

21 (a) Assessment by Director op Defense Re- 

22 SEARCH AND ENGINEERING. — 

23 (1) In geneilvi.. — S ection 139a of title 10, 

24 United States Code, is amended by adding at the 

25 end the following new subsection: 
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1 “((‘)(1) The Director of Defense Research and Engi- 

2 neering shall periodically review and assess the teehno- 

3 logical maturity and integration risk of critical tecli- 

4 nologies of the major defense acquisition programs of the 

5 Department of Defense and report on the findings of such 

6 reviews and assessments to the Under Secretaiy of De- 

7 feiise for Acquisition, Technolog;Sf, and Logistics. 

8 “(2) The Director shall submit to the Secretary of 

9 Defense and to Congress each year a report on the teehno- 

10 logical maturity and integration risk of critical tech- 

1 1 nologies of the major defense acquisition programs of the 

12 Department of Defense.”. 

13 (2) First annum, report. — T he first annual 

14 report under subsection (c)(2) of section 139a of 

15 title 10, United States Code (as added by paragraph 

16 (1)), shall be submitted to Congress not later than 

17 March 1, 2011, and shall address the results of re- 

18 ^iews and assessments conducted by the Director of 

19 Defense Research and Engineering pursuant to sub- 

20 section (c)(1) of such section (as so added) during 

21 the preceding calendar year. 

22 (b) Report on Resources for Implementa- 

23 TION. — Not later than 120 days after the date of the en- 

24 actment of this Act, the Director of Defense Research and 

25 Engineering shall submit to the congressional defense 
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1 committees a report describing: any additional resources 

2 that may be required by the Director, and by other science 

3 and technology elements of the Department of Defense, 

4 to cany out the following: 

5 (1) The requirements under the amendment 

6 made by subsection (a). 

7 (2) The technological maturity assessments re- 

8 quired by section 2366b(a) of title 10, United States 

9 Code, as amended by section 202 of this Act. 

10 (3) The requirements of Department of Defense 

1 1 Instruction 5000, as revised. 

12 SEC. 104. DIRECTOR OF INDEPENDENT COST ASSESSMENT. 

13 (a) Direcjtor <.)p Independent Cost Assess- 

14 MENT. — 

15 (1) In GENERAL. — Chapter 4 of title 10, United 

16 States Code, as amended by section 102 of this Act, 

17 is further amended by inserting after section 139c 

1 8 the following new section: 

19 “§ 139d. Director of Independent Cost Assessment 

20 “(a) There is a Director of Independent Cost Assess- 

21 ment in the Department of Defense, appointed by the 

22 President, by and with the advice and consent of the Sen- 

23 ate. The Director shall be appointed without regard to po- 

24 litieal affiliation and solely on the basis of fitness to per- 

25 form the duties of the Director. 


•S 454 IS 



99 


12 

1 “(b) The Director is the principal ad\lsor to the See- 

2 retaiy of Defense, the Under Secretary of Defense for Ac- 

3 quisition, Technology, and Logistics, and the Under Sec- 

4 retaiy^ of Defense (Comptroller) on cost estimation and 

5 cost analyses for the acquisition programs of the Depart- 

6 ment of Defense and the principal cost estimation official 

7 within the senior management of the Department of De- 

8 fense. The Director shall — 

9 “(1) prescribe, by authority of the Secretary^ of 

10 Defense, policies and procedures for the conduct of 

1 1 cost estimation and cost analysis for the acquisition 

12 programs of tiie Department of Defense; 

13 “(2) provide g-uidance to and consult with the 

14 Secretary of Defense, the Under Secretary of De- 

15 fense for Acquisition, Technology^, and Logistics, the 

16 Under Secretary of Defense (Comptroller), and the 

17 Secretaries of the military departments with respect 

18 to cost estimation in the Department of Defense in 

19 general and with respect to specific cost estimates 

20 and cost analyses to be conducted in connection with 

21 a major defense actiuisition program under chapter 

22 144 of this title or a major automated information 

23 system program under ehairter 144A of this title; 

24 “(3) establish guidance on confidence levels for 

25 cost estimates on major defense acquisition pro- 
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1 grams and require the disclosure of all such eoii- 

2 fideiicc levels; 

3 “(4) monitor and review all cost estimates and 

4 cost analyses conducted in coimeetion with major de- 

5 fense acquisition programs and major automated in- 

6 formation system programs; and 

7 “(5) conduct independent cost estimates and 

8 cost analyses for major defense acquisition programs 

9 and major automated information system pro- 

10 grams — 

11 “(A) in advance of — 

12 “(i) any certification under’ section 

13 2366a or 2366b of this title; 

14 “(ii) any certification under section 

15 2433(e)(2) of this title; and 

16 ' “(iii) any report under section 

17 2445c(f) of this title; and 

18 “(B) whenever necessary to ensure that an 

19 estimate or analysis under paragraph (4) is un- 

20 biased, fair, and reliable. 

21 “(c)(1) The Director may communicate views on mat- 

22 ters within the responsibility of the Director directly to 

23 the Secretarjf of Defense and the Deputy Secretary of De- 

24 fense without obtaining the approval or concurrence of any 

25 other official within the Department of Defense. 
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1 “(2) The Director shall consult closely with, but the 

2 Director and the Director’s staff shall be independent of, 

3 the Under Secretary of Defense for Acquisition, Tech- 

4 nology, and Logistics, the Under Sccretaiy of Defense 

5 (Comptroller), and all other officers and entities of the De- 

6 partment of Defense resj)onsible for acquisition and budg- 

7 eting. 

8 “(d)(1) The Secretary of a military' department shall 

9 report promptly to the Director the results of all cost esti- 

1 0 mates and cost analyses conducted by the mihtary depart- 

1 1 merit and all studies conducted by the military department 

12 in connection with cost estimates and cost analyses for 

13 major defense acquisition jrrogiams of the military depart- 

14 merit, 

15 “(2) The Director may make comments on cost esti- 

16 mates and cost analyses conducted by a military depart- 

17 ment for a major defense acquisition program, request 

18 changes in such cost estimates and cost analyses to ensure 

19 that they are fair and reliable, and develop or require the 

20 development of independent cost estimates or cost anal- 

21 yses for such program, as the Director determines to be 

22 appropriate. 

23 “(3) The Director shall have access to any records 

24 and data in the Department of Defense (including the 

25 records and data of each military' department) that the 
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1 Directoi’ considers necessary to rewew in order to carry 

2 out the Director’s duties under this section. 

3 “(e)(1) The Director shall prepare an annual report 

4 summarizing the cost estimation and cost analysis activi- 

5 ties of the Department of Defense during the previous 

6 year and assessing the progress of the Department in im- 

7 pimlng the accuracy of its costs estimates and analyses. 

8 “(2) Each report under this subs(!etion shall be sub- 

9 mitted concurrently to the Secretary of Defense, the 

10 Under Secretary of Defense for Acquisition, Technology^, 

11 and Logistics, the Und(;r Secretary' of Defense (Comp- 

12 trolhir), and Congress not later than 10 days after the 

13 transmission of the budget foi' the next fiscal year under 

14 section 1105 of title 31. The Director shall ensure that 

15 a report submitted under this subsection does not include 

16 any information, such as proprietary' or source selection 

17 sensitive information, that could undermine the integrity 

18 of the acquisition process. 

19 “(3) The Secretary' may comment on any report of 

20 the Director to Congi'ess under this subsection. 

21 “(f) The President shall include in the budget trans- 

22 mitted to Congress pursuant to section 1105 of title 31 

23 for each fiscal year a separate statement of estimated ex- 

24 penditures and proposed appropriations for that fiscal 

25 year for the Director of Independent Cost Assessment in 
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1 carrying out the duties and responsibilities of the Director 

2 under this section. 

3 “(g) The Seeretarj' of Defense shall ensure that the 

4 Director has sufficient professional staff of military and 

5 civilian personnel to enable the Director to carry out the 

6 duties and responsibilities of the Director under this sec- 

7 tion.”, 

8 (2) CleeicMj aaie.'jd.ment. — T he table of sec- 

9 tions at the beginning of chapter 4 of such title, as 

10 so amended, is further amended by inserting after 

11 the item relating to section 139c the following new 

1 2 item: 

“.139d. DirwUif of Iiide|ieiidcnt (.dist Assossincnt.”. 

13 (b) Report on Monitorino of Operating and 

14 Support Costs for MDAFs. — 

15 (1) Report to secretary op defense. — Not 

16 later than one year after the date of the enactment 

17 of this Act, the Director of Independent Cost Assess- 

18 ment under section 139d of title 10 United States 

19 Code (as added by subsection (a)), shall review exist- 

20 ing systems and methods of the Department of De- 

21 tense for tracking and assessing operating and sup- 

22 port costs on major defense acquisition progi-ams 

23 and submit to the Secretary of Defense a report on 

24 the finding and recommendations of the Director as 

25 a result of the review. 
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1 (2) TkjVnsmittal to congkess. — Not later 

2 than 30 days after receiving the report required by 

3 paragraph (1), the Secretary shall transmit the re- 

4 port to the congressional defense committees, to- 

5 g'cther with any comments on the report the Sec- 

6 retarj' considers appropriate. 

7 (c) Transfer of Peesonnei, and Functions of 

8 Cost Anuvlysis Improvt^iment Group. — T he personnel 

9 and functions of the Cost Analysis Improvement Group 

10 of the Department of Defense are hereby transferred to 

11 the Director of Independent Cost Assessment under see- 

12 tion 139d of title 10, United States Code (as so added), 

13 and shall report directly to the Director. 

14 (d) Conforming Aaiendments. — 

15 (1) Section 2306b(i)(l)(B) of title 10, United 

16 States Code, is amended by striking “Cost Analysis 

17 Improvement Group of the Department of Defense” 

18 and inserting “Director of Independent Cost Assess- 

19 merit”. 

20 (2) Section 2366a(a)(4) of such title is amend- 

21 ed by striking “has been submitted” and inserting 

22 “has been ayiproved by the Director of Independent 

23 Cost Assessment”. 

24 (3) Section 2366b(a)(l)(C) of such title is 

25 amended by striking “have been developed to exe- 
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1 cute” and inserting “have been approved by the Di- 

2 rector of Independent Cost Assessment to provide 

3 for the execution of”. 

4 (4) Section 2433(e)(2)(B)(iii) of such title is 

5 amended by striking “are reasonable” and inserting 

6 “have been determined by the Director of Inde- 

7 pendent Cost Assessment to be reasonable”. 

8 (5) Subparagraph (A) of section 2434(b)(1) of 

9 such title is amended to read as follows: 

10 “(A) be prepared or approved by the Di- 

1 1 rector of Independent Cost Assessment; and”. 

12 (6) Section 2445e(f)(3) of such title is amended 

13 by striking “are reasonable” and inserting “have 

14 been determined by the Director of Independent 

15 Cost Assessment to bo reasonable”. 

16 SEC. 105. ROLE OF THE COMMANDERS OF THE COMBATANT 

17 COMMANDS IN IDENTIFYING JOINT MILITARY 

18 REQUIREMENTS. 

19 Section 181(d) of title 10, United States Code, is 

20 amended — 

21 (1) by inserting “(1)” before “The Under Sec- 

22 retary”; and 

23 (2) by adding at the end the following new 

24 paragraph: 
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1 “(2) The Council shall seek and consider input from 

2 the commanders of the combatant commands in cariying’ 

3 out its mission under paragraphs (1) and (2) of subsection 

4 (b) and in conducting periodic re\aevvs in accordance with 

5 the requirements of subsection (e).”. 

6 TITLE II— ACQUISITION POLICY 

7 SEC. 201. CONSIDERATION OF TRADE-OFFS AMONG COST, 

8 SCHEDULE, AND PERFORMANCE IN THE AC- 

9 QUISITION OF MAJOR WEAPON SYSTEMS. 

10 (a) Consideration op Tipvde-Opps. — 

11 (1) In oeneraIj. — The Secretaiy of Defense 

12 shall develop and implement mechanisms to ensure 

13 that trade-offs between cost, schedule, and perform- 

14 ance are considered as part of the process for devel- 

15 oping requirements for nuyoi' weapon sj^stems. 

16 (2) Ele.me.nts. — The mechanisms required 

17 under this subsection shall ensure, at a minimum, 

18 that — 

19 (A) Department of Defense officials re- 

20 sponsible acquisition, budget, and cost esti- 

21 mating functions are piwdded an appropriate 

22 opportunity to develop estimates and raise cost 

23 and schedule matters before performance re- 

24 quirements are estabhshed for major weapon 

25 sj^tems; and 
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1 (B) consideration is given to fielding major 

2 weapon systems through incremental or spiral 

3 acquisition, while deferring technologies that 

4 are not yet mature, and capabilities that are 

5 likely to significantly increase costs or delay 

6 production, until later increments or spirals. 

7 (3) Major weapons system defined. — In 

8 this subsection, the term “major weapon system” 

9 has the meaning given that term in section 2379(d) 

10 of title 10, United States Code. 

11 (h) Duties of Joint Requirements Ov'ersight 

12 Council. — Section 181(b)(1) of title 10, United States 

13 Code, is amended — 

14 (1) in subparagraph (A), by striking “and” at 

15 the end; 

16 (2) in subparagraph (B), by striking the period 

17 at the end and inserting and”; and 

18 (3) by adding at the end the following new sub- 

19 paragraph: 

20 “(C) in ensuring the consideration of 

21 trade-offs among cost, schedule and perform- 

22 ance for joint militaiy requirements;”. 
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1 SEC. 202. PRELIMINARY DESIGN REVIEW FOR MAJOR DE- 

2 FENSE ACQUISITION PROGRAMS. 

3 Section 2.366b(ci) of title 10, United States Code, is 

4 amended — 

5 (1) in paragraph (1), by striking “and” at the 

6 ejid; 

7 (2) by redesignating paragraph (2) as para- 

8 graph (3); 

9 (3) by inserting after paragraph (1) the fol- 

10 lowing new paragraph (2): 

11 “(2) has received a preliminarj' design rertew 

12 (PDR) and conducted a fonnal post-preliminary de- 

13 sign review assessment, and certifies on the basis of 

14 such assessment that the program demonstrates a 

15 high likelihood of accomplishing its intended mis- 

16 sion; and”; and 

17 (4) in paragraph (3), as redesignated by para- 

18 graph (2) of this section — 

19 (A) in .subparagraph (D), by striking the 

20 semicolon and inserting as determined by the 

21 Milestone Decision Authority on the basis of an 

22 independent review and a.ssessment by the Di- 

23 rector of Defense Research and Engineering; 

24 and”; 

25 (B) by striking subparagraph (E); and 
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1 (C) by redesignating subparagraph (F) as 

2 subparagraph (E). 

3 SEC. 203. MAXIMIZATION OF COMPETITION THROUGHOUT 

4 THE LIFE CYCLE OF MAJOR DEFENSE ACQUI- 

5 SITION PROGRAMS. 

6 (a) ^L\XIMIZATlON OF Competition. — The Sec- 

7 rotary' of Defense shall ensure that the acquisition plan 

8 for each major defense acquisition program includes meas- 

9 ures to maximize competition at both the prime contract 

10 level and the subcontract level of such program throughout 

1 1 the life cj'cle of such program. 

12 (b) Measures To Maximize Competition. — The 

13 measures to maximize competition utilized for purposes of 

14 subsection (a) may include, but are not limited to, meas- 

15 ures to achieve the folloiving, where cost-effective: 

16 (1) Competitive prototyihng. 

17 (2) Dual-sourcing. 

18 (3) Funding of a second source for iiiterchange- 

19 able, next-generation prototype systems or sub- 

20 sj''stems. 

21 (4) Utilization of modular, open architectures 

22 to enable competition for upgrades. 

23 (5) Periodic competitions for subsy^stem up- 

24 grades. 

25 (6) Licensing of additional suppliers. 
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1 (7) Requirements for Government oversight or 

2 approval of make or buy decisions to ensure com- 

3 petition at the subsystem level 

4 (8) Periodic system or program reviews to ad- 

5 dress long-term competitive effects of program deci- 

6 sions. 

7 (9) Consideration of competition at the sub- 

8 contract level and in make or buy decisions as a fac- 

9 tor in proposal evaluations. 

10 (c) C0MPETiTi\T5 PROTOTYPING. — The Secretaiy of 

1 1 Defense shall modify the acciuisition regulations of the De- 

12 partmeiit of Defense to ensure with respect to competitive 

13 prototyping for major defense acquisition programs the 

14 following: 

15 (1) That the acquisition strategy' for each major 

16 defense acquisition progi'um protides for two or 

17 more competing teams to produce prototyi>es before 

18 Milestone B approval (or Key Decision Point B ap- 

19 proval in the case of a space program) unless the 

20 milestone decision authority^ for such program wnives 

21 the requirement on the basis of a determination 

22 that, but for such waiver, the Department would be 

23 unable to meet critical national security objectives. 

24 (2) That if the milestone decision authority 

25 Avaives the requirement for protofyqies produced by 
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1 two or more teams for a major defense aequisitioii 

2 program under paragraph (1), the acquisition strat- 

3 egy for the program provides for the production of 

4 at least one prototj'j)e before Milestone B approval 

5 (or Key Decision Point B approval in the case of a 

6 space program) unless the milestone decision author- 

7 ity waives such requirement on the basis of a deter- 

8 mination that, but for such waiver, the Department 

9 would be unable to meet critical national security ob- 

10 jectives. 

11 (3) That whenever a milestone decision author- 

12 ity authorizes a waiver under paragraph (1) or (2), 

13 the waiver, the determination upon which the waiver 

14 is based, and the reasons for the determination are 

15 submitted in writing to the congressional defense 

16 committees not later than 30 days after the waiver 

17 is authorized. 

18 (d) Major Defense Acquisition PiiOGRAii Dn- 

19 FINED. — In this section, the term “major defense acquisi- 

20 tion program” has thi; meaning given that term in section 

21 2430 of title 10, United States Code. 

22 (c) Applicability. — This section shall apply to any 

23 acquisition plan for a major defense acquisition program 

24 that is developed or reidsed on or after the date that is 

25 60 days after the date of the enactment of this Act. 
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1 SEC. 204. CRITICAL COST GROWTH IN MAJOR DEFENSE AC- 

2 QUISITION PROGRAMS. 

3 (a) Authorized Actions in Event op Critical 

4 Cost Growth. — Section 2433(e)(2) of title 10, United 

5 States Code, is amended — 

6 (1) by redesifftiating subparagraph (C) as sub- 

7 paragraph (D); 

8 (2) by striking subparagraph (B); and 

9 (3) by inserting after subparagraph (A) the fol- 

10 lowing new subparagraphs (B) and (C): 

11 “(B) terminate such acquisition program, un- 

12 less the Seeretaiy determines that the continuation 

13 of such program is essential to the national security 

14 of the United States and submits a written eertifi- 

15 cation in accordance with subparagraph (C)(i) ac- 

16 eompanied by a report setting forth the assessment 

17 carried out pursuant to subparagraph (A) and the 

18 basis for each determination made in accordance 

19 mth clauses (I) through (IV) of subparagraph 

20 (C)(i), together with suj (porting documentation; 

21 “(C) if the program is not terminated — 

22 “(i) sirbmit to Congiess, before the end of 

23 the 60-day period beginning on the day the Se- 

24 lected Acquisition Keport containing the infor- 

25 mation described in subsection (g) is reiiuired 
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to be submitted under section 2432(f) of this 
title, a written certification stating that — 

“(I) such acquisition program is es- 
sential to national security; 

“(II) there are no alternatives to such 
acquisition program which will provide 
equal or gi-eater capability to meet a joint 
military requirement (as that term is de- 
fined in section 181(g)(1) of tliis title) at 
less cost; 

“(III) the new estimates of the pro- 
gram acquisition unit cost or procurement 
unit cost were arrived at in accordance 
with the requirements of section 139d of 
this title and are reasonable; and 

“(15^) the management straeture for 
the acquisition program is adequate to 
manage and control program acquisition 
unit cost or procurement unit cost; 

“(ii) rescind the most recent Milestone ap- 
proval (or Key Decision Point approval in the 
case of a space program) for such program and 
v^dthdi’aw any associated certification under sec- 
tion 2366a or 2366b of this title; and 
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1 “(iii) require a new Milestone approval (or 

2 Key Decision Point approval in the ease of a 

3 space program) for such program before enter- 

4 ing into a new contract, exercising an option 

5 under an existing contract, or otherwise extend- 

6 ing the scope of an existing contract under such 

7 program; and”. 

8 (h) Total Exi^endtture for Procurement Re- 

9 STILTING IN Treatment as MDAP. — Section 2430(a)(2) 

10 of such title is amended by inserting “, including all 

11 planned increments or spiriils,” after “an eventual total 

12 expenditure for procurement”. 

13 SEC. 205. ORGANIZATIONAL CONFLICTS OF INTEREST IN 

14 THE ACQUISITION OF MAJOR WEAPON SYS- 

15 TEMS. 

16 (a) Revised Regui.ations Required. — Not later 

17 than 180 daj’S after the date of the enactment of this Act, 

18 the Under Secretary of Defense for Acquisition, Teeh- 

19 nology, and Logistics shall revise the Defense Supplement 

20 to the Federal Acquisition Regulation to address organiza- 

21 tional conflicts of interest by contractors in the acquisition 

22 of major weapon systems. 

23 (b) EI;EMENTS. — The revised regulations required by 

24 subsection (a) shall, at a minimum — 
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1 (1) ensure that the Department of Defense re- 

2 eeives advice on sj-^tems architecture and sj^stems 

3 engineering matters with respect to major w'eapoii 

4 systems from federally funded research and develop- 

5 ment centers or other sources independent of the 

6 prime contractor; 

7 (2) require that a contract for the performance 

8 of systems engineering and technical assistance 

9 (SETA) functions with regard to a major weapon 

10 system contains a provision prohibiting the con- 

1 1 tractor or any affiliate of the contractor from having 

12 a direct financial interest in the development or con- 

13 straction of the w'eapon system or any component 

14 thereof; and 

15 (3) pro\’ide for fair and objective “make-buy” 

16 decisions by the prime contractor on a major weapon 

17 system bj^— 

1 8 (A) requiring prime contractors to give full 

19 and fair consideration to qualified sources other 

20 than the prime contractor for the development 

21 or construction of major subsystems and com- 

22 poiients of the weapon system; 

23 (B) providing for government oversight of 

24 the process by which prime contractors consider 

25 such sources and determine whether to conduct 
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1 such development or eonstruetiou in-house or 

2 through a subcontract; 

3 (C) where appropriate, requiring that pro- 

4 gram managers, rather than prime contractors, 

5 make the determination whether such develop- 

6 ment or construction should be conducted in- 

7 house or through a subcontract; and 

8 (D) proriding for the consideration of 

9 prime contractors “make-buy” decisions in past 

10 performance evaluations. 

11 (c) Organizational Conflict of Interest Rh- 

12 VIEW Board. — 

13 (1) Est.vblisiimbnt required. — Not later 

14 than 90 days after the date of the enactment of this 

15 Act, the Secretary of Defense shall establish within 

16 the Department of Defense a board to be known as 

17 the “Organizational Conflict of Interest Review' 

18 Board”. 

19 (2) Duties. — The Board shall have the fol- 

20 lowing duties; 

21 (A) To advise the Under Seeretarv' of De- 

22 fense for Acquisition, Technology, and Logistics 

23 on policies relating to org-anizational conflicts of 

24 interest in the acquisition of major w'eapon sys- 

25 terns. 
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1 (B) To achlse program managers on steps 

2 to comply with the requirements of the revised 

3 regulations required by this section and to ad- 

4 dress organizational conflicts of interest in the 

5 acquisition of major weapon systems. 

6 (C) To advise appropriate officials of the 

7 Department on organizational conflicts of inter- 

8 est arising in proposed mergers of defense con- 

9 tractors. 

10 (d) Major Weapon System Defined. — In this sec- 

11 tion, the term “major weapon system” has the meaning 

12 given that term in section 2379(d) of title 10, United 

13 States Code. 

14 SEC. 206. AWARDS FOR DEPARTMENT OF DEFENSE PER- 

15 SONNEL FOR EXCELLENCE IN THE ACQUISI- 

1 6 TION OF PRODUCTS AND SERVICES. 

17 (a) In Gener^vl. — Not later than 180 days after the 

1 8 date of the enactment of this Act, the Secretary of Defense 

19 shall commence cariying out a program to recognize excel- 

20 lent performance by individuals and teams of members of 

21 the Armed Forces and civilian personnel of the Depart- 

22 ment of Defense in the acquisition of products and serv'- 

23 ices for the Department of Defense. 

24 (b) Elements. — The program required by sub- 

25 section (a) shall include the following: 
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1 (1) Procedures for the nomination by the per- 

2 sonnel of the military departments and the Defense 

3 Agencies of individuals and teams of members of the 

4 Ai-med Forces and civilian personnel of the Depart- 

5 ment of Defense for eligibility for recognition under 

6 the program. 

7 (2) Procedures for the evaluation of nomina- 

8 tions for recognition under the program by one or 

9 more panels of individuals from the government, 

10 academia, and the private sector who have such ex- 

11 pertisc, and are appointed in such manner, as the 

12 Secretary shall establish for purposes of the pro- 

13 gram. 

14 (e) Award of Cash Bonusks. — ^A s part of the pro- 

15 gram required by subsection (a), the Secretarj^ may award 

16 to any individual recognized pursuant to the program a 

17 cash bonus authorized by any other piwdsion of law to 

1 8 the extent that the performance of such individual so rec- 

19 ognized warrants the award of such bonus under such pro- 

20 vision of law. 

O 
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March 14, 2009 

Ken Krieg 

711 Xavier Court 

McLean, VA 22101 


Mr. Peter Levine 
General Counsel 
Committee on Armed Services 
United States Senate 
Washington, DC 20510-6050 

Dear^^^f^ 

Thank you and Senators Levin and McCain for the opportunity to comment on the Weapon Systems 
Acquisition Reform Act of 2009. I enjoyed catching up with you a couple of weeks ago and hope the 
conversation was useful. I will leave the bulk of my comments to that discussion. But, as promised, I 
have tried to capture my thoughts about the proposal for an Independent Cost Assessment office. As I 
told you then, I concur with the objectives, but recommend a different course of action for now; always 
reserving the right to force a significant organizational change at a later date. 

We shared a set of similar objectives: 

• Get risk fully addressed and valued in decisions, 

• Make the milestone decision maker trade between reducing risk elements and increasing the 
available budget to the most likely cost, and 

• Making risk decisions stick over time. 

Success in the objectives requires both analytic capability and commitment of decision makers to use it. 
The problems we agreed on include: 

• There are not enough good cost analysts in the system; especially in the services, 

• There are competing purposes, even inside the cost community, between just doing better cost 
estimates and really exposing and increasing the understanding of risk inherent in programs, 

• Decision makers vary greatly in their belief in full budgeting of risk (pendulum may be swinging 
but there will not be universal consensus), 

• The enterprise (DoD and Congress) including everyone involved in requirements, budgeting and 
risk management (acquisition) does not stay disciplined to decisions (even those well made) 
over time. There should be pressure for both configuration control and financial (capital) 
discipline if you want to make good risk decision making stick over time), and 

• We are stuck with the runoutof decisions for a longtime after they are made. 
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As both the Director of Program Analysis and Evaluation and as the Under Secretary for Acquisition, 
Technology and Logistics, I managed and used the Cost Analysis and Improvement Group (CAIG) and 
independent cost analysis in decisions. ! am a big fen of their work and have a caution of trying not to 
do unintentional harm while trying to fix something. 

From my experience, significant reorganizations take time to work through. The proposal to pull the 
CAIG out of PA&E would be a significant reorganization; the CAIG uses the resources of the rest of the 
PA&E staff quite effectively. Pulling them out into a stand-alone oi^anization would, in my opinion, 
cause more disruption than add value. 

In addition, there is real value in the candid discussion and advice that the CAIG director now provides to 
the USD(AT&L) if the Under Secretary values it. Pulling the organization out of PA&E and making it a 
widely reporting organization risks the effectiveness of that candor -again totally in my opinion. 

Finally, Improving the use of cost estimating in DoD to support the better management of risk requires 
more work in the services than in the office of the secretary of defense. The independent cost capability 
in the services was significantly reduced over time. ft»e ability of the services to conduct "should cosf' 
analyses was a victim of that reduction. 

Based upon these opinions and logic, I would propose the following direction - for now: 

• Keep the CAIG where it is and concentrating on Major Defense Acquisition Programs in 
milestones and Nunn-McCurdy reviews; 

• Give the Director of the CAIG functional responsibility for the cost community in DoD; the CAIG 
would: 

o Set standards, approve and distribute toot sets, etc. 
o Review human capital initiatives in the cost community 
o Evaluate performance of other cost organizations 

• Direct the Military Departments to rebuild their cost staffs to concentrate on ACAT 2s and 3s 
and provide should cost analyses for programs 

• Ask the Department to advise the Congress on the degree to which its programs are budgeted 
to independent cost estimates. 

You can reserve the creation of an independent cost office for another year. 

Peter, I offer these thoughts in the spirit of trying to advance the objectives while minimizing 
unintended consequences. I wish you the best of luck. 


Sirwrerely,^' 
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Written Remarks of to the Senate Armed Services Committee Re^ardinf^ Defense Acquisiti 


Chairman Levin, Ranking Member McCain, and distinguished members of the committee, thank 
you for the opportunity to submit written remarks on behalf of the Congressional Research 
Service (CRS) to be included in the hearing record. As requested, this statement examines 
acquisition reform and options that Congress may choose to consider to improve the defense 
acquisition system. Specifically, this statement focuses on 

• the Nunn-McCurdy Act as a mechanism for tracking and managing cost growth. 

• unrealistic cost estimates as a driver of cost growth. 

• programs that have high cost growth risks. 


The Nunn-McCurdy Act as a Mechanism for Tracking and 
Managing Cost Growth 

For 25 years, the Nunn-McCurdy Act has served as one of the principal mechanisms for notifying 
Congress of cost overruns in Major Defense Acquisition Programs. The act’s origins can be 
traced back to May 14, 1981, when Senator Sam Nunn rose to the floor of the Senate and offered 
a floor amendment requiring the Department of Defense (DOD) to notify Congress if, in Fiscal 
Year (FY) 1982, total program acquisition unit costs for any Major Defense Acquisition Program 
(MDAP) under certain circumstances, exceeded 15% or 25%.^ The amendment was included in 
the Department of Defense Authorization, 1982.’ 

That same year, Representative Dave McCurdy, then chairman of the House Armed Services 
Committee Special Panel on Defense Procurement Procedures, held a series of hearings 
examining weapon system cost growth.’* Together, Senator Nunn and Representative McCurdy 
successfully led the effort to pass the Nunn-McCurdy Act, which perpetuated the requirement 
that DOD notify Congress when cost overruns exceed certain thresholds.’ 

Nunn-McCurdy Provides Congress Improved Visibility into Cost Growth 

The Nunn-McCurdy thresholds are based on a comparison between a program’s actual costs and 
the current baseline estimate* or the original baseline estimate.’ As shown in Table 1, a 


' A Major Defense Acquisition Program (MDAP) is a program estimated to require research, development, test, and 
evaluation (RDT&E) costs of more than $365 million or procurement costs of more than $2,190 billion (inFY 2000 
constant dollars). In 1983, the procurement cost of a program had to be one billion dollars (in FY1980 constant dollars) 
to be considered an MDAP. 

^ Specifically, the amendment required DOD to notily Congress if total program acquisition unit costs for any MDAP 
for which no procurement funds were authorized increased by more than 15%, and total program acquisition unit cost 
for any MDAP or current unit acquisition cost increased by more than 25%. See; Congressional Record lAay 14th, 1981 
pg. S. 5014-15. 

^ 95 Stat. 1 129. See: Congressional RecordMsy 14ih, 1981 pg. S. 5016. 

“ U S. Congress, House Armed Services Committee, Special Panel on Defense Procurement Procedures, House Armed 
Services Hearings,^ q\. 1 1, 97th Cong., Istsess. (Washington: GPO, 1981), p. 1. 

^ See, Department of Defense Authorization Act, 1983 (96 Stat. 718). 

^ A current baseline estimate is defined as the cost of the attire program, including research, development, test, & 
evaluation (RDT&E), and procurement. See: 10 U.S.C. § 2435(a). The baseline estimate is part of the Acquisition 
Program Baseline (APB). The APB “contains the most important cost, schedule, and performance parameters (both 
objectives and thresholds) for the program. It is ^proved by the Milestone Decision Authority (MDA), and signed by 
(continued...) 
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Written Remarks of to the Senate Armed Services Committee Re^^ardin^ Defense Acquisiti 


“significant” Nunn-McCurdy breach occurs when the Program Acquisition Unit Cost (PAUC) or 
the Procurement Unit Cost (PUC) increases 15% or more over the current baseline estimate or 
30% or more over the original baseline estimate.* A “critical” breach occurs when the PAUC or 
PUC increases 25% or more over the current baseline estimate or 50% or more over the original 
baseline estimate. 


Table I . Nunn-McCurdy Breach Thresholds 



Significant Breach 

Critical Breach 

Current Baseline Estimate 

>15% 

>25% 

Original Baseline Estimate 

>30% 

>50% 

Source: 10 U.S.C. § 2433, 


When a Nunn-McCurdy breach occurs, DOD must submit a Selected Acquisition Report (SAR) 
to Congress, which includes cost, schedule, and perfonnance information.^ In the event of a 
critical breach, the Secretary of Defense must assess the program to determine the cause of the 
cost growth, project the cost of completing the program (with and without requirement 
modifications), and assess reasonable alternatives to the existing system. After the reassessment, 
for the program to continue uninterrupted, the Secretary of Defense must certify to Congress that 
the program is essential to national security; there is no viable alternative to the program; the new 
cost estimate is reasonable; and the management structure is sufficient to control cost growth. 

Nurm-McCurdy Does Not Require Reporting on Operations & Support Costs 

The Nunn-McCurdy Act does not apply to all elements of a weapon system’s life-cycle costs. For 
example, the Act does not apply to costs incurred during operations, support, or disposal. 

Analysts have estimated that Operations & Support (O&S) costs account for two-thirds or more 
of a system’s total life-cycle cost. Weapon systems have experienced O&S cost growth as well 


(...continued) 

the program manager (PM) and his/her direct chain of supervision...” See: 
[competitivesourcing.navy.mil/rcfercnce_documcnts/dcfs.cfmj. Last visited Nov. 6, 2008. 

’An original baseline estimate is defined as the baseline estimate established prior to the program entering “system 
development and demonstration”, also knows as “Milestone B”. or at program initiation, whichever occurs later. See: 

10 U.S.C.§ 2435(d). 

* PAUC is defined as the total cost of development, procurement, and construction divided by the number of units. 

PUC is defined as the total procurement cost divided by the number of units. Program acquisition unit cost and 
procurement unit cost are defined in 10 U.S.C. § 2432(a). 

^ 10 U.S.C. § 2433. A Selected Acquisition Report “reports the status of total program cost, schedule, and performance, 
as well as program unit cost and unit cost breach information. For joint programs, the SAR reports the information by 
participant. Each SAR will include a full, life-cycle cost analysis for the reporting program, each of its evolutionary 
increments, as available, and for its antecedent program, if applicable.” See Defense Acquisition Guidebook, 
[https://akss.dau.mi1/DAG/Guideboolc/IG_clO.9.2.asp]. Last visited Nov. 7, 2008. 

Operations and support costs are funded from Military Personnel, Operations and Maintenance, Procurement, and 
occasionally RDT&E appropriations. 

‘ ' Walt Cooper, O&S Trends and Current Issues, Office of the Secretary of Defense, Cost Analysis Improvement 
Group, Washington, D.C,, May 2007. See also, Office of the Under Secretary of Defense For Acquisition. Technology, 
and Logistics, Report of the Defense Science Board on Developmental Test & Evaluation, Washington, D.C., May 

2008, p. 22. 
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as acquisition cost growth. Cost growth in O&S costs can reduce the funds available to acquire 
new or upgrade existing weapon systems. Given tiie costs associated with operations and support. 
Congress may want to consider applying Nunn-McCurdy type reporting requirements to O&S 
costs. This might entail changes in DOD’s cost estimating procedures and other elements of the 
acquisition process. 

Many of the decisions that determine O&S costs are made early in the acquisition process — 
prior to significant O&S costs being incurred. Because O&S costs are not incurred until much 
later in the life-cycle, these costs may not always get the same attention as acquisition costs at 
Milestone B (the engineering and manufacturing development and demonstration phase) and 
Milestone C (the production and deployment phase). Decisions made at these key decision points 
could result in lower acquisition costs at the expense of higher long term O&S costs — and 
ultimately higher overall life-cycle costs. 

DOD has made efforts to improve how it tracks and estimates O&S costs. DOD and each military 
service tracks O&S costs through the Visibility and Management of Operating and Support Costs 
(VAMOSC) program. And in October 2007, the Office of the Secretary of Defense, Cost Analysis 
Improvement Group also issued an Operating and Support Cost Estimate Guide. Applying 
Nunn-McCurdy type of reporting requirements to O&S costs might help Congress set its 
budgetary priorities as well as gather and track cost data for future analysis. Another option for 
Congress would be to require an independent cost assessor to include in an annual report to 
Congress a comparison of original O&S cost estimates to current actual costs (adjusted for 
inflation) for ongoing programs. The extent to which these options may be viable depends on the 
reliability of the data available. Without good cost data on O&S costs, DOD and Congress may 
not have important information upon which to make budget decisions. 

Nunn-McCurdy as a Mechanism for Managing Cost Growth 

The Nunn-McCurdy Act, (10 U.S.C. 2433) is basically a “reporting requirement for programs” 
experiencing cost overruns, and is generally not structured to help manage programs or allocate 
funds.'^ Early proponents of Nunn-McCurdy believed that requiring DOD to report on cost 
growth would suffice because public exposure of cost overruns would force DOD to rein in cost 
growth in weapons programs. Senator John Tower, then Chairman of the Senate Armed Services 
Committee, expressed a somewhat different view, stating more than 25 years ago that notifying 
Congress only after cost growth had occurred is like “closing the gate after the horse has galloped 
off into the boondocks.”*^ 


Office of the Secretary of Defense, Cost Analysis Improvement Group, Operating and Support Cost Estimating 
Guide, October 2007. 

Sec [http;//www.eia.org/news/pressreleases/2006-04-26.268.phtmll. Last visited 12/23/2008. 

As Representative McCurdy explained, “The assumption behind the Nunn-McCurdy provision of the fiscal 1983 
defense authorization bill was that the prospect of an adverse reaction from the Office of Management and Budget, 
Congress, or the public would force senior Pentagon officials to address the question of whether the program in 
question - at their newly reported, higher costs - were worth continuing.” See 
[http;//www.eia.org/new3/pressreleases/2006-04-26.268.phtm!]. Last visited 1 2/23/2008. 

Congressional Recordyizy 14th, 1 981 pg. S. 5012. 
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Generally, a Nunn-McCurdy breach does not result in a program being cancelled. However, there 
have been some exceptions. For example, in July 2008, Congress was notified that the Armed 
Reconnaissance Helicopter (ARH) program had suffered a critical Nunn-McCurdy breach. In 
response, John Young, Under Secretary of Defense for Acquisition, Technology, and Logistics, in 
consultation with senior Army officials, cancelled the ARH program. Secretary of the Army Pete 
Geren justified the cancellation, stating that “The cost and schedule that were the focus of the 
decision to award the contract to Bell Helicopter are no longer valid. We have a duty to the Army 
and the taxpayer to move ahead with an alternative course of action to meet this critical capability 
for our soldiers at the best price and as soon as possible.”'^ 

In spite of Nunn-McCurdy and other defense acquisition reform efforts, DOD acquisition 
programs have continued to experience cost growth. GAO recently reported that 44% of 
programs in the FY2007 portfolio experienced acquisition unit cost growth of at least 25%, 
compared with 37% of programs experiencing such cost growth in the FY2000 portfolio.” GAO 
also reported that the total acquisition costs for MDAPs in the FY2007 portfolio increased 26% 
from first cost estimates, compared with a 6% increase in 2000.'* 

Some analyses show that the size of cost overruns have grown over time and may continue into 
the foreseeable future.’’ According to a RAND study, over the past 50 years, annual cost 
escalation rates for amphibious ships, surface combatants, attack submarines, and nuclear aircraft 
carriers have ranged from 7% to 1 1%, and the annual cost escalation rate for U.S. fighter aircraft 
was about 1 0%.’° The combination of increased costs for individual systems and increased cost 
growth make it more difficult for DOD to develop an acquisition strategy that stays within budget 
while simultaneously allowing the United States to maintain the forces that it needs to meet future 
challenges. 

Some analysts argue that acquisition reform may not succeed until policies are adopted that 
consistently punish programs or contractors for failing to perform. Some of these analysts have 
called for strengthening Nunn-McCurdy as a vehicle to manage MDAPs. 


Department of Defense, ’’DoD Announces Non-Certification Of Armed Reconnaissance Helicopter Program," press 
release, October 16, 2008, http://www.defenselink.mil/releases/release.aspx?releaseid=12288. 

'Mbid. 

Government Accountability Office. Defense Acquisitions: Assessments of Selected Weapons Programs. GAO-08- 
467SP. March 31, 2008. Highlights Page. 

For example, one analysis found that “cost overruns are increasing by an average of 1.86 percentage points per year. 
If this trend is allowed to continue, the analysis suggests that in 10 years the average overrun will exceed 56 percent....” 
See: Deloitte Consulting LLP, Can We Afford Our Own Future? WhyA&D Programs are Late and Over-budget — 
and What Can Be Done to Fix the Problem, 2008, p. 2. 

Mark V. Arena, Irv Blickstein, and Obaid Younossi, et al., Why Has the Cost of Navy Ships Risen? A Macroscopic 
Examination of the Trends in U.S. Naval Ship Costs Over the Past Several Decades, RAND Corporation, 2006, p. xiv. 

For example, Clark Murdock of the Center for Strategic and International Studies, has been quoted as calling for a 
“Nunn-McCurdy on steroids that really punishes programs that have failed.” See: Christopher J. Castelli, "DEFENSE: 
Acquisition Shop Prepares for Shift in Administrations," Inside Missile Defense , November 5, 2008, Vol. 14, No. 23. 
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Unrealistic Cost Estimates as a Driver of Cost Growth and Nunn- 
McCurdy Breaches 

For some 30 years, various DOD officials, analysts, and industry officials have argued that a 
primary cause of cost growth in DOD acquisitions is unrealistically low cost estimates.^^ 
Unrealistically optimistic cost estimates can make future cost growth almost inevitable. Michael 
Gilmore of the Congressional Budget Office recently stated when discussing overly optimistic 
cost estimates, “no program manager in the world is going to be able to manage the program in 
such a way that the costs will not grow... it’s not really so much cost growth as cost realism 
setting in.”^^ Such unrealistically low cost estimates set the stage for future Nunn-McCurdy 
breaches. 

In 2006, Gary Payton, Air Force Deputy Under Secretary for Space Programs, made a direct link 
between unrealistically optimistic estimates and Nunn-McCurdy breaches. In a presentation 
entitled Nunn-McCurdys Aren 't Fun, he argued that “Unbridled optimism regarding cost, 
schedule, performance, and risks is a recipe for failure.”^'* As set forth in the presentation, 

Understated costs leads to lower budget — » leads to industry bidding price less than budget 
— »■ leads to lower award price — ♦ leads to government repeatedly changing scope, schedule, 
budget profile ^ leads to five to ten years later recognition “real” cost multiple of bid — > 
leads to Nunn-McCurdy Breach. 

GAO has also linked optimistic cost estimates with significant cost growth, finding that “the 
Navy tends to underestimate the costs needed to construct ships — resulting in unrealistic budgets 
and large cost increases after ship construction has begun.”^^ 


Why Cost Estimates Are Sometimes Unrealistically Optimistic 

Senior Defense officials, past and current, acknowledge that program advocates have strong 
incentives to underestimate program acquisition costs. Contractors use low cost estimates to win 
the contract; program representatives use low estimates to argue for approval of the system 
against competing systems.^® In 1981, then Deputy Secretary of Defense Frank C. Carlucci, 
testified, low cost estimates “are fueled by optimistic contractor proposals to win competitions 


Poor cost estimating was a recurring theme during the McCurdy hearings. For example, then Director of the Program 
Analysis and Evaluation Office, Maj. Gen. Patrick M. Roddy staled that there are three fundamental cost growth 
drivers, inflation, poor cost estimating, and scheduling. GAO stated “Cost estimating is probably the key ingredient in 
reducing cost growth. ..As far back as the early 1970’s, GAO has reported that both planning and development cost 
estimates on Federal acquisitions in many cases are quite optimistic.. .unrealistically low contractor and agency 
estimates on the front end aggravates cost growth. What is needed is more candor up front in presenting programs to 
the Congress and not promising more than can be realistically delivered.” And then Deputy Secretary of Defense Frank 
C. Carlucci, in a written statement to Congress, stated that “early cost, schedule, and performance estimates are overly 
optimistic”. See: House Anned Services Hearings, 97th Cong., 1st Sess., Volume II, 1981. Op. Cit. P. 74, 1009, and 
1085, respectively. 

U.S. Congress, House Committee on the Budget, Long-Term Sustainability of Current Defense Plans, 1 1 1th Cong., 
1st sess., February 4, 2009. 

See [http://www.dtic.mil/ndia/2006systemsAVednesday/payton.pdf], p. 10, Last visited December 23, 2008. 

Government Accountability Office. DEFENSE ACQUISITIONS: Realistic Business Cases Needed to Execute Navy 
Shipbuilding Programs. GAO-07-943T. July 24, 2007, p. 17, 

See; House Armed Services Hearings, 97th Cong., 1st Sess., Volume 11, 1981. Op. Cit. p. 883. 
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and program managers who want to see their programs fiinded.”^^ In a memo dated January 30, 
2009, John Young echoed this sentiment, stating “the enterprise will often pressure acquisition 
teams and industry to provide low, optimistic estimates to help start programs.”^* 

There seems to be little disincentive for contractors or program representatives to use unrealistic 
cost estimates. With some notable exceptions, programs rarely get cancelled as a result of cost 
growth. Michael J. Busch, Director, Contracts and Proposals for Williams International, stated at 
the 1981 McCurdy hearings, “L. .would not give contracts to contractors who had cost overruns of 
substantial size that were attributed to poor cost and schedule management. ..No, I would not 
continue to give contracts to somebody who can’t perform. Many analysts agree, arguing that 
cost estimates will generally continue to be unrealistically low until a policy is adopted that 
consistently punishes programs — and contractors — for relying on unrealistic cost estimates. 
Given the contraction in the defense industrial base and the limited number of companies able to 
produce certain weapons systems, some analysts questions how contractors could be effectively 
punished for failing to use realistic cost estimates or stay within budget. 

Injecting More Realism in Cost Estimates and Strengthening Nunn-McCurdy 

The extent to which cost estimates are useful depends in part on the reliability of the cost 
estimates. The absence of reliable cost estimates may put Congress in the position of having to 
decide on budget priorities with potentially inadequate information upon which to base its 
decisions. One method of evaluating the reliability of a cost estimate is to develop a confidence 
level, which is generated by conducting uncertainty analyses that measure the probability of cost 
growth. Programs witli a 50 percent confidence level have a 50 percent chance of staying within 
the estimated cost; programs with an 80 percent confidence level have an 80 percent chance of 
staying with the estimated costs. Another option Congress may consider would be to obtain the 
confidence levels of MDAP cost estimates and ensure that the confidence levels are developed 
using best practices. 

Another option that Congress may consider would be to require an independent cost assessor to 
establish guidance on generating and requiring the disclosure of confidence levels and monitor 
cost estimates and analyses performed by DOD. Congress may also consider requiring an 
independent cost assessor to include in an annual report to Congress a comparison of DOD cost 
estimates and confidence levels, independent cost estimates and confidence levels, and actual 
MDAP costs. Such a report could provide Congress with a single source for measuring the 
effectiveness of cost estimates and confidence levels in predicting MDAP costs. 

Relying on Cost Estimates and a Knowledge-Based Approach for Key Decision 
Points 

Another factor contributing to poor cost estimates is basing such estimates on inadequate 
knowledge as a result of immature technologies or design. John Young wrote in a recent memo 


House Armed Services Hearings, 97‘^ Cong., I** Sess., Volume H, 1981. Op. Cit. p. 1086. 

John J. Young, Jr., Reasons for Cost Changes for Selected Major Defense Acquisition Programs (MDAPs), 
Memorandum, January 30, 2009. 

“ House Armed Services Hearings, 97"' Cong., 1“ Sess., Volume 11, 1981. Op. Cit. p. 653. 
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that low initial estimates are made based on inadequate knowledge. Some analysts argue that as 
weapon systems have become increasingly complex, a knowledge-based approach has become 
more critical to developing realistic cost estimates. GAO recently reported that in approximately 
60% of the programs it assessed, DOD lacked the knowledge necessary to “reasonably estimate 
the cost and time it would take to develop and produce the product.”^' 

Milestone B is when programs enter engineering and manufacturing development. According to a 
RAND study, cost growth for completed programs was 46% from Milestone B, compared to 16% 
from Milestone C, when programs enter the production and deployment phase. John Young 
recently stated that a major driver of cost growth is programs passing milestone B with 
inadequate knowledge, including immature technologies, immature design, and fluid 
requirements.^^ Some of the programs he identified include 

1 . Joint Strike Fighter (insufficient understanding of design), 

2. V-22 (immature technology), 

3. Family of Medium Tactical Vehicles (design flaws), and 

4. LPD-1 7 (flawed lead ship design process). 

Allowing programs to proceed through Milestone B and other key points without sufficient 
knowledge increases the likelihood that cost estimates will be unrealistically optimistic and that 
programs will fail to stay on budget and on schedule. 

One option Congress may consider is to require that MDAPs complete a preliminary design 
review (PDR) and a post-PDR assessment prior to Milestone B approval and that technological 
maturity and integration risks be rigorously assessed earlier in the acquisition process. The extent 
to which such options may prove to be effective may hinge, in part, on the extent to which 
programs will not be allowed to proceed if they fail to meet established benchmarks. As discussed 
above, John Young acknowledged that a major driver of cost growth is prematurely allowing 
programs to pass through key decision points and program reviews. 

Other Drivers of Cost Growth 

A number of other factors are drivers for cost growth in defense acquisitions. These factors 
include programs not getting stable funding, contracts that are awarded without full and open 
competition, insufficient testing early in the acquisition process, requirements creep, and poor 
contractor management and oversight. Each of these factors can undermine a program’s ability to 
execute on time and within budget. For example, adding requirements to a program after a cost 
estimate has been completed will likely undermine the realism of the estimate because the 
assumptions upon which the estimate is based are no longer valid. Similarly, early developmental 


John J. Young, Jr., Reasons far Cost Changes for Selected major Defense Acquisition Programs (MDAPs), 
Memorandum, January 30, 2009. 

Government Accountability Office. DEFENSE ACQUISITIONS: Realistic Business Cases Needed to Execute Navy 
Shipbuilding Programs. GAO-07-943T, July 24, 2007, p, 5. 

Mark V. Arena, Obaid Younossi, and Lionel A. Galway, et al., Impossible Cenainty Cost Risk Analysis for Air 
Force, RAND National Defense Research Institute, 2006, p. xx. 

John J. Young, Jr., Reasons for Cost Changes for Selected major Defense Acquisition Programs (MDAPs), 
Memorandum, January 30, 2009. 
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testing can identify flaws in a system early on in development. Generally, it is cheaper to identify 
and fix problems early in the acquisition process. The proposed bill seeks to address some of 
these issues by requiring DOD reestablish the position of Director of Developmental Testing and 
requiring military departments to assess their developmental testing capabilities. These 
requirements could result in a more robust testing capability earlier in the acquisition process. 
Such early testing may provide decision makers with critical information at key decision points. 


Programs That Have High Cost Growth Risks 

Congress may consider putting in place a process for providing a higher level of scrutiny for 
programs that are considered high risk programs. Secretary of Defense Robert Gates testified that 
big-ticket items are the primary driver of overall cost growth, stating that “five programs account 
for more than half of total cost growth.’^** Among the programs that he identified as having 
problems were those that either involved new technologies or were among the most expensive, 
including the V-22 Osprey, Future Combat Systems, and Joint Strike Fighter. 

One category that could be considered high risk is those programs that involve significant leaps in 
technology. For example, the components or complimentary programs of the Future Combat 
System that have experienced significant cost growth, such as the Joint Tactical Radio System 
and Warfighter Information Network-tactical, contained the most significant leaps forward in 
technology. And the VH-7 1 presidential helicopter had a Nunn-McCurdy breach, in part, because 
it sought to incorporate dramatically improved capabilities. 

Another category that could be considered high risk is DOD’s most expensive programs; even 
moderate cost growth on high-dollar programs can jeopardize other DOD acquisition programs. 
For example, an average of 10% cost growth in DOD’s ten most expensive acquisition programs 
would amount to cost overruns of approximately $19 billion, or 1 3.5% of the funding required to 
execute the remaining 85 programs (see Table 2). 


Table 2. Planned RDT&E and Procurement Funding for MDAPs 

(in Billions) 



FY2008 

FY2009 

FY2010 

FY20 1 1 

FY20 1 2 

Total 

Funding for Top 1 0 MDAPs 

$39.1 

$40.6 

$37.3 

$35.2 

$42.0 

194.2 

Funding for remaining 85 MDAPs 

J33.2 

$31.5 

$26.9 

$25.4 

$24.1 

141. 1 

Top 10 MDAPs as a percentage of total funding 

54% 

56% 

58% 

58% 

64% 

58% 


Source: Government Accountability Office. 

Notes: Data as of December 2006. Figures in FY2008 dollars. 


Given the impact that cost growth on high-risk or high-value programs can have on DOD’s 
overarching acquisition strategy, one option for Congress would be to require DOD to take a 
more cautious risk-management approach to such programs. For example, Congress could require 
DOD to budget for certain programs at a higher confidence level, an option that has been raised in 
various forms by a number of observers, including the Defense Acquisition Performance 


U.S, Congress, Senate Committee on Armed Services, Challenges Facing [he Department of Defense, 11 1th Cong., 
1 St sess-, January 27, 2009, p. 1 1 . 
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Assessment Panel and GAO.^^ Alternatively, DOD and an independent cost assessor could be 
required to submit to Congress, concurrent with a budget request, not only the confidence level 
for the current cost estimate, but what the cost estimate would be at an 80% confidence level.^^ 
Providing Congress with a range of cost estimates and associated confidence levels may give 
Congress more insight into the a program’s potential cost growth-and potential risk. 

An additional option that Congress may consider would be to require high-risk programs to 
pursue a more conservative development process. For example, such programs could generally 
more strictly enforce a requirement that programs pursue a spiral development strategy, an 
approach that may reduce the risks associated with the development of weapons systems 
containing a number of new technologies. Another option would be to require high-risk programs 
to achieve a high-level of technology development before being approved for Milestone B. As Air 
Force Secretary Michael Donely was recently quoted as saying, “too many of our weapon 
programs take on high levels of technical risk in development. ..[we must] “take a more cautious 
approach to new starts.”^’ 


Author Contact Information 


Moshe Schwartz 

Analyst in Defense Acquisition 

mschwartz@crs.loc.gov, 7-1463 


See Lieutenant General Ronald Kadish, USAF (Ret), Dr. Gerald Abbott, and Mr. Frank Cappuccio, et al,, Defense 
Acquisition Performance Assessment Report, Defense Acquisition Performance Assessment Project, January 2006, p, 
13; U.S. Government Accountability Office, Defense Acquisitions: Realistic Business Cases Needed to Execute Navy 
Shipbuilding Programs, GAO-07-943T, July 24, 2007, p. 17. 

Using an 80% level was proposed by the Defense Acquisition Performance Assessment Report. 

Marina Malenic, "USAF Must Take More Cautious Approach To New Weapon Programs, Top Official Says," 
Defense Daily, Februaiy 27, 2009, p. 3. 
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April 17, 2006 

MEMORANDUM FOR UNDER SECRETARY OF DEFENSE FOR ACQUlSmON, 
TECHNOLOGY. AND LOGISTICS 
UNDER SECRETARY OF DEFENSE FOR PERSONNEL 
AND READINESS 

AUDITOR GENERAL, U.S. ARMY AUDIT AGENCY 


SUBJECT: Report on the DoD Acquisition Workforce Count 
(Report No. D-2006-073) 


We are providing this report for review and comment. We received comments on 
a draft of this report from the Director, Human Capital Initiatives, Office of the Under 
Secretary of Defense for Acquisition, technology, and Logistics, the Director, Army 
Contracting .Agency; the Director, Defense Manpower Data Center; and the Military 
Deputy, Office of the Assistant Secretaiq/ of the Air Force (Acquisition). We considered 
the management comments when preparing the final report. 

DoD Directive 7650.3 requires that all recommendations be resolved promptly. 
Defense Management Data Center comments to Recommendation .A.2. were partially 
responsive. As a result of managemenl comments from the office of the Under Secretary 
of Defense for Acquisition, Technology, and Logistics, we renumbered draft report 
Recommendation A.l.b(2) as final report Recommendation A.3., and redirected it to the 
Under Secretary of Defense for Personnel and Readiness to respond in coordination with 
the Office of the Under Secretary of Defense for Acquisition Technology and Logistics. 
We request Defense Management Data Center comments on Recommendation A. 2. and 
Under Secretary of Defense (Personnel and Readiness) comments on 
Recommendation A.3. by June i, 2006. 

If possible, please send management comments in electronic format (Adobe 
.Acrobat file only) to Aiidcm@.dodiij.mil . Copies of the management comments must 
contain the actual signature of the authorizing official. We cannoi accept the i Signed / 
symbol in place of the actual signature. If you arrange to send classified comments 
eiectronicaliy, they must be sent over the SECRET Internet Protocol Router Network 
(SIPRNET).” We appreciate the courtesies extended to the staff. Questions should be 
directed to Ms. Kimberley A. Caprio at (703) 604-9202 (DSN 664-9202) or 
Mr. Benjamin A. .Mehlman at (703) 604-9291 (DSN 664-9291). See Appeiidi.x E for the 
report distribution. The team members are listed inside the back cover. 

By direction of the Deputy Inspector General for Auditing: 



Richard B. JUTTiTCe 
Assistant Inspector General 
Acquisition and Contract Management 
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DoD Acquisition Workforce Count 


Executive Summary 


Who Should Read This Report and Why? Service acquisition executives, prograin 
managers, DoD contracting officials, and personnel with interest in the DoD acquisition 
workforce should read this report. This report addresses the DoD acquisition workforce 
count and congressional reporting as well as related contract administration and 
surveillance. 

Background. The DoD acquisition workforce is subject to two methods of counting and 
congressional reporting. The first method, known as the “Acquisition Organization” 
workforce counting approach, was used by the 1986 President’s Blue Ribbon 
Commission on Defense Management (the Packard Commission) and counts all 
personnel employed in 22 designated DoD acquisition organizations as part of the 
acquisition workforce, regardless of an employee’s occupation. As of September 30, 
2004, there were 206,653 civilian and military personnel included in the 22 designated 
DoD acquisition organizations. The second acquisition workforce counting method is 
known as the “Refined Packard” approach. In May 1997, the Office of the Under 
Secretary of Defense for Acquisition, Technology, and Logistics contracted with 
Jefferson Solutions, a division of Jefferson Consulting Group, to review alternative ways 
of identifying the acquisition workforce. In September 1 997, Jefferson Solutions 
reported to the Office of the Under Secretary a proposed Refined Packard acquisition 
workforce methodology of combining occupational and organizational data for 
identifying those in the workforce. In a December 18,1 997, letter, the Secretary of 
Defense forwarded the Jefferson Solutions report to Congress and stated that, beginning 
October 1, 1998, members of the acquisition workforce would be uniformly identified 
using the Refined Packard model. While accepting the new approach, the House Arrned 
Services Committee requested that DoD continue to report the Acquisition Organization 
workforce count in conjunction with the Refined Packard count. 

As of September 30, 2004, there were 134,602 civilian and military personnel included in 
the DoD Refined Packard count. Of the 134,602 personnel in the Refined Packard 
workforce count, 69 percent (92,588) were assigned and included in the DoD Acquisition 
Organization count, while 31 percent (42,014) of the DoD Refined Packard workforce 
were assigned outside the 22 major DoD acquisition organizations. The Refined Packard 
model removed 55 percent (1 14,065) of the 206,653 personnel included in the 
Acquisition Organization count because those personnel perform non-acquisition support 
functions, such as firefighting, police, human resources, administration, accounting, 
legal, engineering technicians, supply, transportation, and trades (such as equipment and 
facilities operations and maintenance). 

Results. DoD annually reports to Congress a count of its acquisition workforce as 
compiled by the Defense Manpower Data Center through application of the Refined 
Packard method. The FY 2004 and prior Refined Packard workforce counts were 



unverifiable. As a result, DoD acquisition workforce planning risks, including for the 
Quadrennial Defense Review, could increase because annual Refined Packard workforce 
support and expenditures may be based on unreliable data and may not accurately reflect 
the true DoD acquisition workforce (finding A). Defense Manpower Data Center 
management controls for the acquisition workforce counting process were not adequate 
to ensure the count reflects the true DoD acquisition workforce. Implementing our 
recommendations will improve Defense Manpower Data Center acquisition workforce 
counting procedures. 

The Defense Contracting Command-Washington did not properly negotiate or administer 
support service contract DASW01-03-F-0393 with Jefferson Solutions. In addition, the 
contracting officer representative approved Jefferson Solutions monthly invoices without 
requesting written support of services performed, hours expended, or products provided. 
As a result, the Government is not assured that fair and reasonable prices were negotiated 
with Jefferson Solutions, nor that work performed by Jefferson Solutions in identifying 
and reporting the Refined Packard workforce was done properly for the time and 
materials expended or that Government resources were used economically (finding B). 

Recommendations. We recommend that the Director, Defense Procurement and 
Acquisition Policy develop and implement written standard operating procedures and 
guidance for defining and counting the acquisition workforce, methodologies used to 
perform periodic workforce counts, and requirements to maintain and support acquisition 
workforce count documentation; and revise DoD Instruction 5000.55 to ensure consistent 
acquisition workforce information format and reporting from the DoD Components. We 
also recommend that the Under Secretary of Defense for Personnel and Readiness in 
coordination with the Under Secretary of Defense for Acquisition, Technology, and 
Logistics revise the instruction to estimate contractor equivalents that support the DoD 
acquisition workforce, and include such estimates as supplementary annual DoD 
acquisition workforce reporting data to Congress. We recommend the Director, Defense 
Manpower Data Center develop a knowledge management program to maintain corporate 
knowledge of Defense Manpower Data Center information systems and processes. We 
recommend that the Commander, Defense Contracting Command-Washington ensure 
that acquisition workforce count contracts supporting Office of the Under Secretary of 
Defense for Acquisition, Technology, and Logistics components require written 
contractor monthly progress reports. 

Management Comments and Audit Response. Management concurred with 
recommendations to develop and implement written standard operating procedures and 
guidance for defining and counting the acquisition workforce, methodologies used to 
perform periodic workforce counts, and requirements for acquisition workforce count 
documentation; and to revise DoD Instruction 5000.55 to ensure consistent acquisition 
workforce information format and reporting from the DoD Components. Management 
also concurred with the recommendation to ensure that acquisition workforce count 
contracts supporting the Office of the Under Secretary of Defense for Acquisition, 
Technology, and Logistics components require written contractor monthly progress 
reports. We request by June 1 , 2006, the Under Secretaiy of Defense for Personnel and 
Readiness comment regarding the redirected recommendation to revise DoD 
Instruction 5000.55 to estimate contractor equivalents that support the DoD acquisition 
workforce, and that the Director, Defense Manpower Data Center provide additional 
comments regarding the recommendation to develop a knowledge management program 
to maintain corporate knowledge of Defense Manpower Data Center information systems 
and processes. 
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Background 


During the last 1 5 years, DoD has experienced significant challenges and expectations 
regarding its acquisition workforce due to declining number of workers, while the 
workload and the demand for technical expertise and skill sets are increasing. The 
National Defense Authorization Act for Fiscal Year 1996, section 906 (Public 
Law 104-106) implements a plan to restructure the DoD acquisition organization by 
reducing the number of civilian and military personnel employed or assigned to 
acquisition organizations by 25 percent over a 5-year period and to eliminate duplication 
of functions among existing acquisition organizations. However, from FY 1999 through 
FY 2004, the total DoD procurement dollars increased 78 percent (from $135 billion to 
$241, billion) and total DoD procurement actions increased 14 percent (from 5.8 million 
to 6.6 million). 

Strategic Planning for the DoD Acquisition Workforce. On September 27, 2005, the 
Under Secretary for Defense for Acquisition, Technology, and Logistics (USD[AT&L]) 
stated in testimony to the Senate Committee on Armed Services that his office initiated a 
comprehensive review of the acquisition workforce. It will have a DoD acquisition 
workforce strategic plan in place no later than 120 days after the completion of the 
Quadrennial Defense Review. The USD(AT&L) stated that the strategic plan would 
incorporate the National Security Personnel System and would be aligned with 
Quadrennial Defense Review results and analysis of the current acquisition workforce 
and evolving workload requirements, such as service contracts and contingency 
operations. The USD(AT&L) noted that a thoughtful acquisition workforce strategic 
plan would define processes and tools to assess workforce capability and to tactically 
recruit, develop, and retain the right talent, with emphasis on smart execution and 
implementation. 

The USD(AT&L) noted that the DoD acquisition workforce has had to contend from 
FY 1998 through FY 2004 with a 105 percent increase in constant DoD contracting 
dollars and a 58 percent increase in contracting actions over $100,000, which are often 
the most complex. The USD(AT&L) noted that the average age of the DoD civilian 
workforce is 46.7 years old and the number of workforce members with 30 or more years 
of experience continues to increase. Therefore, DoD will face losing a significant 
amount of acquisition workforce knowledge, experience, and capabiiity. The 
USD(AT&L) also expressed concern about an impending talent gap created by a 10-year 
acquisition workforce drawdown, but noted that DoD has taken measures to alleviate the 
talent gap through Defense Acquisition University development of increased on-line and 
on-site acquisition training and the establishment of a Web-based acquisition Community 
of Practice system. 

Defense Acquisition Workforce Improvement Act. The 1990 Defense Acquisition 
Workforce Improvement Act (section 1701-1764, title 10, United States Code) (the Act). 
The Act was enacted to improve the effectiveness of the civilian and military acquisition 
workforce through enhanced education, training, and career development, and thereby 
improve the acquisition process. The Act requires DoD to establish formal career paths 
for those people who want to pursue careers in acquisition. DoD has accomplished this 
by dividing acquisition positions into 1 1 categories and establishing a formal certification 
process. The Act requires each of the Military Services to designate a Director, 
Acquisition Career Management (DACM) to oversee implementation of the Act’s 
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requirements. The Defense Acquisition University is required to designate a “Fourth 
Estate” DACM to oversee implementation of the Act’s requirements at Defense agencies 
and DoD field activities. 

Acquisition Workforce Counting Methods. Acquisition workforce counts support 
DoD acquisition workforce planning and estimating expenditures for workforce training 
and development, as well as annual congressional reporting. The Senate and House 
Armed Services Committees used two formats to count the DoD acquisition workforce. 
The first method, known as the “Acquisition Organization” workforce counting 
approach, counts all personnel employed in 22 designated DoD acquisition organizations 
as part of the acquisition workforce, regardless of an employee’s occupation. See 
Appendix C for a list of the 22 acquisition organizations. TTie second method is known 
as the “Refined Packard,” or the “Acquisition, Technology, and Logistics” workforce 
counting approach, which combines occupational and organizational data for identifying 
employees in the acquisition workforce. 

Workforce Counting Process. The Office of the USD(AT&L) has designated the 
Defense Manpower Data Center (DMDC) as the lead office for conducting annual 
acquisition workforce counts. A May 13, 1999, USD(AT&L) policy memorandum, 
“Refined Packard Key Acquisition and Technology Workforce Identification Policy for 
the Fiscal Year 1999,” designates the DMDC as the official database for the acquisition 
workforce personnel counts. The Military Services and the Defense agencies submit 
quarterly acquisition workforce count data to DMDC. The data are stored in a DMDC 
data repository. DMDC reviews, revises, and augments the data submissions by breaking 
out the submission by pay grades, certifications, and civilian and military components. 
For the Refined Packard count, DMDC is assisted by the AT&L contractor Jefferson 
Consulting Group (Jefferson Solutions), which creates an annual Refined Packard 
acquisition workforce count report. DMDC also creates an annual Acquisition 
Organization workforce count report. A USD(AT&L) component, the Office of the 
Director, Defense Procurement and Policy, exercises oversight over the workforce 
counting process. 

Military Services and Defense Logistics Agency Acquisition Workforce Counting 
Processes. Guidance and processes used by the Military Services and the Defense 
Logistics Agency (DLA) to count and report the acquisition workforce v^ greatly. All 
three services code their.acquisition positions in individual Military Service personnel 
management information systems. Array acquisition positions are coded by the Office of 
the Director Acquisition Career Management (DACM), while the Navy and Air Force 
major commands code their acquisition positions. The coding creates the databases that 
serve as the basis for both the Acquisition Organization and the Refined Packard counts. 
The Military Services DACM staffs are responsible for the acquisition workforce count 
submitted to DMDC. DMDC makes an assessment of the acquisition workforce count 
and sends results back to the Army, Navy, and Air Force DACM staffs for review and 
correction. Individual Military Service DACM staffs send corrections back to DMDC, 
and DMDC submits the corrected totals for the inclusion to the Military Services Refined 
Packard count. 

The DLA Customer Support Office in Columbus, Ohio, submits quarterly DLA 
Acquisition Workforce count data directly to DMDC. The Customer Support Office 
produces quarterly Acquisition Workforce counts through extraction of DLA Acquisition 
Personnel File records from the Defense Civilian Personnel Data System and use of DLA 
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Acquisition Position Files. DMDC performs a match between the Acquisition Personnel 
File records and Acquisition Position File records; the results of that match become the 
DLA Refined Packard count. See Appendix D for a detailed explanation of the Military 
Services and DLA acquisition workforce counting processes, criteria, and counting 
systems. 

Acquisition, Technology, and Logistics Policy Memorandums. The Office of the 
USD(AT&L) has issued several policy memorandums regarding conduct of annual 
acquisition workforce counts. An April 6, 2001, USD(AT&L) policy memorandum on 
“Assimilation of Newly Identified Personnel into the Acquisition and Technology 
Workforce” describes a process of coding designated acquisition and technology 
positions and personnel by position category career field into the Defense Civilian 
Personnel Data System and comparable military personnel systems. Subsequently, a 
November 21, 2003, policy memorandum on “Moratorium on Designated People and 
Positions as Part of the Defense Acquisition, Technology, and Logistics (AT&L) 
Workforce” imposes a plus or minus 1 percent change restriction on the number of 
Defense acquisition and support personnel to implement requirements of the FY 2004 
National Defense Authorization Act. A November 9, 2005, policy memorandum on 
“Lifting of the Moratorium on Designating People and Positions as Part of the Defense 
Acquisition, Technology, and Logistics (AT&L) Workforce” rescinds the November 21 , 
2003, memorandum, and allows DoD components to resume designation of new 
positions that perform acquisition functions. 

DoD Instruction 5000.55, “Reporting Management Information on DoD Military 
and Civilian Acquisition Personnel and Position,” November 1, 1991. DoD 
Instruction 5000.55 outlines the acquisition workforce personnel data submission 
requirements for both civilian and military personnel. The instruction establishes a 
management information system capable of providing standardized information on 
acquisition positions and on persons serving in acquisition positions. In addition, the 
instruction attempts to create a DoD-wide capability for monitoring, reporting, and 
tracking the composition, education, experience, and training status of the acquisition 
workforce and to establish uniform procedures for submitting manpower, personnel, and 
assignment information on selected DoD acquisition workforce civilian and military 
personnel. The instruction also establishes procedures for reporting functional and 
training-related data on selected DoD civilian and military personnel to evaluate the 
mandatory training requirements and status of the acquisition workforce. DoD 
Instruction 5000.55 designates DMDC as custodian of all automated records collected 
under the instruction and to provide data quality control, inquiry capabilities, and 
administrative and computer support. The instruction also requires DACMs to maintain 
documentation on the identification of DoD civilian and military acquisition positions. 

DoD Directive 5000.52, “Defense Acquisition, Technology, and Logistics Workforce 
Education, Training, and Career Development Program,” January 12, 2005. The 
revised Directive updates policies and responsibilities for an education, training, and 
career development program for the acquisition workforce and establishes a single 
Acquisition Corps throughout DoD. This includes defining the responsibilities of the 
Offices of the Under Secretaries of Defense for Acquisition, Technology, and Logistics; 
Personnel and Readiness; Comptroller; and Intelligence; and for DoD Component 
Heads. The directive implements chapters? of title 10, United States Code on Defense 
Acquisition Workforce information systems, career development programs, and methods 
of identifying Refined Packard workforce positions. 
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DoD Instruction 5000.66, “Operation of the Defense Acquisition, Technology, and 
Logistics Workforce Education, Training, and Career Development Program,” 
December 21, 2005. The instruction implements DoD Directive 5000.52 and provides 
uniform guidance for managing acquisition workforce positions and career development. 
The instruction designates and identifies workforce positions; the attainment and 
maintenance of competencies through education, training, and experience; management 
of the Defense Acquisition Corps; selection and placement of personnel in acquisition 
positions; and workforce metrics. The instruction states that the USD(AT&L) will 
determine uniform policies and procedures for the acquisition workforce education, 
training, and career development programs and implement DoD issuances including a 
Desk Guide for workforce career management. On Janu^ 10, 2006, the Defense 
Acquisition University issued “A Desk Guide for Acquisition, Technology, and Logistics 
Workforce Career Management” (Desk Guide) to provide more detailed program 
guidance. The Desk Guide states that all acquisition workforce positions fall in 1 of the 
15 categories and that positions can include part-time, temporary, and full-time 
Government civilian and military personnel. 


Objectives 


The overall audit objective was to review the effectiveness of the DoD acquisition 
workforce. The specific objective was to review the impact on the DoD acquisition 
workforce of changes in workload requirements, contracting methods, and mandated 
workforce reductions. For purposes of this report, we address how effectively DoD 
identified personnel included in the acquisition workforce. The review of the 
management control program at major acquisition organizations as it relates to the 
acquisition workforce was also an announced objective. 


Managers’ Internal Control Program 

DoD Directive 5010.38, “Management Control (MC) Program,” August 26, 1996, and 
DoD Instruction 5010.40, “Management Control (MC) Program Procedures,” August 28, 
1 996, require DoD organizations to implement a comprehensive system of management 
controls that provides reasonable assurance that programs are operating as intended and 
to evaluate the adequacy of the controls. 

Scope of the Review of the Management Control Program. We reviewed the 
adequacy of DMDC management controls over the acquisition workforce counting 
process. Specifically, we reviewed DMDC management controls over annual acquisition 
workforce count standard operating procedures, including documented analysis or 
reconciliations of quarterly Military Service and Defense agency workforce count 
submissions. We reviewed management’s self-evaluation applicable to those controls. 

We also reviewed the adequacy of Army, Navy, Air Force, and DLA managenient 
controls over the acquisition workforce counting process. Specifically, we reviewed 
management controls over their operating procedures and methodology. Because we did 
not identify a material weakness, we did not assess management’s self-evaluation. 


4 



141 


Adequacy of Management Controls. We identified a material management control 
weakness for DMDC as defined by DoD Instruction 5010.40. DMDC management 
controls for the acquisition workforce counting process were not adequate to ensure the 
count reflects the true DoD acquisition workforce. Implementing recommendations A. 1 . 
and A.2. will improve DMDC acquisition workforce counting procedures. A copy of the 
report will be provided to the senior official responsible for management controls in the 
Office of the Under Secretary of Defense for Personnel and Readiness. 

Army, Navy, Air Force, and DLA management controls over the acquisition workforce 
counting process were adequate as they applied to the audit objective. 

Adequacy of Management’s Self-Evaluation. DMDC did not identify the acquisition 
workforce counting process as an assessable unit; therefore, DMDC did not identify or 
report the material management control weakness identified by the audit. 
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A. DoD Acquisition Workforce Database and 
Counting Controls 

DoD annually reports to Congress the Refined Packard workforce count as 
compiled by the DMDC. The FY 2004' and prior Refined Packard workforce 
counts were unverifiable. The counts were unverifiable because: 

• DMDC lacked standard operating procedures and corporate 
knowledge of past Refined Packard workforce counts, events, and 
requirements; 

• neither DMDC, nor a DoD contractor supporting the count, maintained 
analyses or performed reconciliations of quarterly Military Service and 
Defense agency workforce counts used to create the annual Refined 
Packard workforce count; 

• two DMDC acquisition workforce Management Information Systems 
were only partially operational. 

In addition the Refined Packard and Acquisition Organization counts did not 
include and did not report contract equivalents; and DoD acquisition workforce 
planning risks, including risks for the Quadrennial Defense Review, could 
increase because annual Refined Packard workforce support and expenditures 
may be based on unreliable data and may not accurately reflect the true DoD 
civilian, military, and contracted acquisition workforce. 


DoD Acquisition Workforce 


The DoD acquisition workforce is subject to two methods of counting and congressional 
reporting. The first method is known as the “Acquisition Organization” workforce 
counting approach. The second method is known as the “Refined Packard,” or the 
“Acquisition, Technology, and Logistics,” workforce counting approach. Both methods 
continue to be used as both are requested by congressional oversight committees on an ad 
hoc basis. The audit did not determine the extent of support for workforce counts made 
through the Acquisition Organization method as those counts were derived directly from 
DoD civilian and military personnel databases. 

Acquisition Workforce Count. The Acquisition Organization workforce counting 
approach was used by the 1986 President’s Blue Ribbon Commission on Defense 
Management (Packard Commission). The approach counted all personnel employed in 
22 designated DoD acquisition organizations as part of the acquisition workforce, 
regardless of an employee’s occupation. As of September 30, 2004, there were 206,653 
civilians and military personnel included the 22 designated DoD acquisition 
organizations. The total DoD Acquisition Organization workforce count since F Y 1 999 
is shown in Table 1. 


' FY 2004 Refined Packard count as of September 30 , 2004 , reported in March 2005. 
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Table 1. DoD Acquisition Organization Workforce 
(Excluding Maintenance Depot Civilians) 


Fiscal Year 

1999 

2000 

2001 

2002 

2003 

2004 

Workforce 

230,556 

219,419 

215,090 

212,482 

213,670 

206,653 


DoD does not count DoD civilians assigned to maintenance depots in the Acquisition 
Organization workforce count. AT&L personnel stated that the House Armed Services 
Committee requested that DoD report acquisition workforce levels to the conmiittee 
using the Acquisition Organization format. Of the 206,653 acquisition organization 
personnel counted for FY 2004, 55 percent (1 14,065) were non-acquisition personnel 
performing support functions, such as firefighting, police, human resources, 
administration, accounting, legal, engineering technicians, supply, transportation, and 
trades (such as equipment and facilities operations and maintenance). The remaining 
45 percent (92,588) were performing designated acquisition workforce missions and are 
included in the DoD Refined Packard workforce count as shown in the Figure below. 


Acquisition Organization 
Workforce (Excludes 
Maintenance Depot Workforce) 
Total: 206,653 



Acquisition Organization and Refined Packard Workforce as of September 30, 2004 
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Refined Packard Workforce Count. In May 1997, the Office of the USD(AT&L) 
contracted with Jefferson Solutions to review alternative ways of identifying the 
acquisition workforce instead of using the Acquisition Organization workforce counting 
method. This effort was in response to congressional criticism that DoD lacked a 
consistent, Defense-wide approach for determining both the size of the acquisition 
workforce and the skill sets of those serving in it. In September 1 997, Jefferson 
Solutions reported to AT&L a proposed Refined Packard acquisition workforce 
methodology of combining occupational and organizational data for identifying those in 
the workforce. In a December 18, 1997, letter, the Secretary of Defense forwarded the 
Jefferson Solutions report to Congress and stated that, beginning October 1, 1998, 
members of the acquisition workforce would be uniformly identified using the Refined 
Packard model.^ While accepting the new approach, the House Armed Services 
Committee requested that DoD continue to report both the Acquisition Organization 
workforce count and the Refined Packard workforce count. The Senate Armed Services 
Committee relies on the Refined Packard method. 

Refined Packard Model. The Refined Packard model uses occupations and 
organizations for determining whether an individual will be counted as part of the 
workforce under three categories. The model can be briefly stated as follows: 

• Civilians in Category 1 occupations (such as contracting) are assumed to be 
performing acquisition functions regardless of the organization in which they 
serve. Therefore, all of these individuals across DoD are included in the 
acquisition workforce. 

• Civilians in Category II occupations are counted only when serving in 
organizations that primarily perform acquisition missions, such as the materiel 
commands or technology missions, and research labs. Category II 
occupations cover such job series such as electronics engineering, budget 
analysis, and computer engineering. 

• Military or civilian personnel in Category III occupations can be added to the 
count or removed from the count, at the discretion of the Military Services 
and Defense agencies. For example, military enlisted personnel performing 
key acquisition functions can be added using this capability. 

• Military officers subject to the Defense Acquisition Workforce Improvement 
Act are counted as part of the acquisition workforce. 

As of September 30, 2004, there were 134,602 civilian and military personnel included in 
the Refined Packard workforce count. Of the 134,602 personnel, 69 percent (92,588) 
was assigned and included in the DoD Acquisition Organization workforce count, while 
31 percent (42,014) of the Refined Packard workforce count was assigned outside the 
22 major DoD acquisition organizations. This total compares with a September 30, 2003, 
count of 1 34,43 1 . The comparable Refined Packard workforce counts totaled 1 32,593 
for FY 2002; 129,249 for FY 2001; 135,014 for FY 2000; and 138,851 forFY 1999. The 


^ DoD refers to the acquisition workforce count produced by the “Refined Packard" model as the “Acquisition, 
Technology, and Logistics” count. 
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total acquisition personnel for the Military Services and Fourth Estate^ from FY 1999 
through FY 2004 are shown in Table 2. 


Table 2. Refined Packard Workforce Counts From FY 1999 through FY 2004 

Fiscal 

Year 

Army 

Navy 

Air Force 

Fourth Estate 

Totals 

1999 

39,592 

47,895 

29,258 

22,106 

138,851 

2000 

40,080 

41,373 

28,959 

24,602 

135,014 

2001 

41,074 

37,158 

27,820 

23,197 

129,249 

2002 

41,783 

39,661 

28,444 

22,705 

132,593 

2003 

47,697 

41,622 

27,888 

17,224 

134,431 

2004 

48,251 

41,552 

27,775 

17,024 

134,602 


As shown in Table 3, the Refined Packard count has remained substantially less than the 
corresponding fiscal years’ Acquisition Organization workforce count. 


Table 3. Acquisition Organization and Refined Packard Workforce Counts From 
FY 1999 through FY 2004 

Fiscal Year 

1999 

2000 

2001 

2002 

2003 

2004 

Acquisition 
Organ izatio 
n 

230,556 

219,419 

215,090 

212,482 

213,670 

206,653 

Refined 

Packard 

138,851 

135,014 

129,249 

132,593 

134,431 

134,602 


Verification of Acquisition Workforce Count 


The Refined Packard workforce count for FY 2004 was unverifiable for the following 
reasons. DMDC lacked standard operating procedures and eorporate knowledge of past 
Refined Packard workforce counts, events, and requirements. Neither DMDC nor the 
DoD support contractor, Jefferson Solutions, maintained analyses or performed count 
reconciliations. Neither the Refined Packard workforce count nor the Acquisition 
Organization workforce count included reported contract equivalents. Two DMDC 
acquisition workforce Management Information Systems were only partially operational. 

Annual Workforce Count Procedures and Corporate Knowledge. DMDC lacked 
standard operating procedures and corporate knowledge of past Refined Packard 
workforce counts, events, and requirements. DMDC lacked processes and written 
procedures for compiling and reporting the annual DoD-wide count of the acquisition 


^ Fourth Estate is composed of the acquisition workforce for the Defense Contract Management Agency, Defense 
Logistics Agency, Defense Information Systems Agency, Missile Defense Agency, Defense Contract Audit 
Agency, and other DoD agencies and field activities. 
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workforce. Although DMDC and Jefferson Solutions performed analyses of workforce 
data provided by the Military Services and the Defense agencies, no files or records were 
maintained by DMDC, Jefferson Solutions, or the Military Services to document the 
scope of FY 2004 or prior year reviews of Military Service or Defense agency Refined 
Packard submissions. DMDC management stated they were not required to maintain 
standard operating procedures or written documentation for compiling and reporting the 
DoD acquisition workforce count. 

DMDC also lacked any personnel with corporate knowledge of how Refined Packard 
workforce counts were conducted and reported prior to FY 2004. While Jefferson 
Solutions personnel maintained corporate knowledge of Refined Packard counting 
methodology, Jefferson Solutions did not document or maintain standard operating 
procedures of past Refined Packard workforce counts, events, and requirements. All key 
DMDC personnel involved in the workforce counting process either retired or left the 
agency from 1997 through 2003, effectively leaving new staff for 2004. DMDC did not 
maintain a knowledge management program^ to allow replacement personnel to gain 
experience or background on the program. 

Analysis of Workforce Counts. Neither DMDC nor Jefferson Solutions maintained 
analyses or reconciliations of quarterly Military Service and Defense agency workforce 
counts used to create the annual Refined Packard workforce count. The FY 2004 and 
prior Refined Packard workforce counts were unverifiable. For example, DMDC could 
not explain nor provide written documentation why there were significant increases and 
decreases for acquisition workforce personnel for the Military Services and Defense 
agencies from FY 1999 through FY 2004. DMDC also did not maintain documentation 
for review and correction of the acquisition workforce counts with the Military Services 
and Defense agencies. DMDC personnel relied upon input by Jefferson Solutions 
personnel to analyze the FY 2004 Refined Packard workforce count. Because Jefferson 
Solutions did not maintain detailed records of yearly workforce count analyses, DMDC 
lacked sufficient audit trails for analyzing and reconciling quarterly Military Service and 
Defense agency workforce submissions. 

DMDC Acquisition Workforce Management Information Systems. The 2004 
“DMDC Profile” handbook listed two information systems used to support the annual 
acquisition workforce counting process. However, as of January 2006, neither 
management information system was fully operational or reliable. As a result, the 
DMDC lacked the management information system infrastructure to support its 
Acquisition Organization and Refined Packard workforce counts. 

Defense Acquisition Workforce Improvement Act Management Information 
System. The Defense Acquisition Workforce Improvement Act Management 
Information System was a title given by DMDC for its use of four defense personnel 
systems to extract acquisition workforce information to meet the Defense Acquisition 
Workforce Improvement Act and DoD Instruction 5000.55 reporting and training 
requirements. The Defense Acquisition Workforce Improvement Act Management 
Information System was not an integrated management information system, but rather a 


^ Knowledge management is a systematic approach to finding, understanding, and using knowledge to achieve 
organizational objectives. Many organizations are developing tools, systems, and awareness among employees 
that capturing and sharing knowledge is an important oiganizational practice. Knowledge management creates 
value when knowledge most important to the organization is shared and reused. 
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“wrapper” for DMDC use of the four existing systems used to produce the Acquisition 
Organization and Refined Packard workforce counts. It could not provide a trace of 
actions performed by DMDC or the Military Services in producing or reconciling annual 
workforce counts. DMDC management stated the management information system was 
not meant to be an integrated system. 

Defense Acquisition Workforce Management Information System. The 
Defense Acquisition Workforce Management Information System included a 
nonoperational Web-based Data Mart application to be used for DoD Instruction 5000.55 
workforce reporting purposes. The Data Matt application was to store, report, and 
summarize personnel data elements, position information, and training-related 
information supporting the requirements associated with the Defense Acquisition 
Workforce Improvement Act. If DMDC can make the Data Mart application operational, 
it will provide a means to support an auditable trace to the annual workforce counts. The 
management information system also included an operational “Outreach” application to 
allow the Office of the Secretary of Defense and Defense Acquisition University users to 
create, tailor, and use a list of DoD Acquisition Workforce e-mail addresses for 
educational and informational purposes. The Outreach application does not support 
Acquisition Organization and Refined Packard workforce counts. 

Use of Contracted Acquisition Workforce Support Services. Neither Acquisition 
Organization nor the Refined Packard workforce counts included or reported contractor 
full-time equivalents’ (contractor equivalents) used to support acquisition activities. The 
Militaty Services and DoD agencies are supplementing acquisition workforce shortfalls 
with contracted acquisition workforce personnel including procurement, engineering, 
program management, and supply support. Although DoD is not required to report the 
number of contractor equivalents, omitting contractors from the workforce count results 
in the invisibility of a large part of the true acquisition workforce. Table 4 shows the 
number and percentage of contractor equivalents at six DoD locations visited. 

Contracted acquisition workforces are not required to meet Defense Acquisition 
Workforce Improvement Act training and certification requirements. In some cases 
contracted personnel may be integrated into the Government acquisition workforce 
structure performing the same duties as the Government personnel. 

Reasons for Use of Contracted Acquisition Support. According to focus group^ 
interviews conducted as a part of the audit at the six acquisition workforce locations, 
increased use of contracted acquisition support occurred because of increased workload 
coupled with past reductions of acquisition workforce personnel. Focus ^oup comments 
also indicated that prioritizing acquisition and contracting job responsibilities led to the 
identification of severable functions that could be contracted out. Contracted acquisition 
support personnel were used to meet these priority skill sets. The acquisition workforce 
shortfalls were prevalent across several acquisition career fields including program 
management, quality assurance/engineering, and contracting. 


^ Per the DoD A-76 Costing Manual of March 14, 2001, a civilian fuil-time equivalent position is generally 
considered equal to 1,776 hours of annual productive effort. 

* Focus groups included program managers, quality assurance/engineeis, and contracting personnel. 
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Table 4. Acquisition Workforce Contractor Equivalents for Locations Visited 

Location 

Visited 

DoD 

Civilians 

DoD 

Military 

Contractor 

Equivalents 

Total 

Government 
Acquisition 
Workforce + 
Contractor 
Equivalents 

Contractor 
Percentage 
of Combined 
Acquisition 
Workforce 

Army 

Acquisition 

Support 

Center 

85 

15 

22 

122 

1 8 percent 

Naval Sea 

Systems 

Command 

14,040 

453 

Not 

Determinable 

Not 

Determinable 

Not 

Determinable 

Air Force 
Aeronautica 

1 Systems 
Center 

2,265 

797 

1,133 

4,195 

27 percent 

Air Force 
Space and 
Missile 
Systems 
Center 

514 

632 

2,053 

3,199 

64 percent 

Defense 

Supply 

Center 

Columbus 

712 

0 

133 

845 

16 percent 

Defense 

Supply 

Center 

Richmond 

798 

0 

250 

1,048 

24 percent 

*The Naval Sea Systems Command could not provide data on the number ot contractor equivalents. 


Program Management. Program managers from the focus groups indicated 
program offices heavily rely on contracted support personnel because their offices are not 
authorized or do not have the persotmel resources to hire additional Government 
personnel to meet current workload requirements. The Government lacked the ability to 
compete with private industry for experienced acquisition workforce positions in certain 
metropolitan regions, including Washington, D.C.; Los Angeles; and Boston, For 
example. Air Force Personnel at the Space and Missile Systems Center in Los Angeles, 
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California, noted that Federally Funded Research and Development Centers and 
contracted system engineering and technical assistance support are being used in “hard- 
to-compete” job locations. Other comments noted regional Government personnel hiring 
freezes caused by budget restrictions as a primary restriction. 

Quality Assurance/Engineering. Quality assurance and engineering personnel 
from the focus groups indicated that quality assurance workforce reductions created 
increased workload requirements for engineering personnel. For example, several sites 
noted that system safety engineers (who concentrate on system failures and risk 
management operations) have been significantly reduced from the acquisition workforce, 
while transferring the additional workload and responsibilities to remaining Government 
engineering personnel. Government personnel at the Naval Sea Systems Command, the 
Air Force Aeronautical Systems Center, and the Space and Missile Systems Center noted 
the quality assurance function was being performed by engineers. Personnel stated that 
the lack of Government resources tends to increase program delivery date noncompliance 
issues because the Government is not able to identify and fix system failures prior to 
delivery. 

Contracting. Both Government contracting workforce and contracting managers 
noted that contracting personnel have experienced increased workload requirements 
because of post-9/1 1 procurement demands. The Government contracting workforce and 
contracting mangers also stated deployments of contracting personnel to overseas 
locations are increasing and that deployed personnel positions' are not being backfilled, 
causing shortages of Government personnel and greatly increasing workloads of 
warranted contracting officers. Contracting personnel at all locations visited indicated 
contract close-out procedures have been generally outsourced as contracted acquisition 
support services. Contracting managers generally noted that contract close-out was 
considered one area where Government contracted for support services to compensate for 
Government acquisition personnel shortages. 


Effect of Potential Inaccurate Acquisition Workforce Count 


DoD acquisition workforce planning risks could increase because annual Refined 
Packard workforce support and expenditures may be based on unreliable data. A lack of 
standard operating procedures for workforce counts and requirements have created 
conditions that may lead to a lack of consistency and comparability of acquisition 
workforce data throughout recent years including FY 2004. A lack of corporate 
knowledge, including prior analyses and reconciliations of quarterly Military Service and 
Fourth Estate workforce counts may contribute to a lack of understanding. DoD annual 
reporting to Congress should provide more specific information about Acquisition 
Organization and Refined Packard workforce count changes. Information provided 
should explain work force deletions, newly created positions, and shifts of personnel 
from non-Refined Packard workforce to Refined Packard workforce positions, as well as 
shifts between different Refined Packard workforce position categories. 


’ Deployments included transfer of both Government civilian and military personnel to forward locations, as well as 
Military Reserve/Nationa! Guard call-ups of Government civilians to active duty. 
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Six acquisition offices reviewed reported contractors comprising 1 6 to 64 percent of the 
combined Govemment/contractor acquisition workforce. Contracted acquisition 
workforce equivalents are not required to meet Defense Acquisition Workforce 
Improvement Act training and certification requirements, as are their DoD civilian and 
military counterparts. As a result, the need to track the number of contractor equivalents 
has increased. Thus, DoD should revise Instruction 5000.55 to estimate and track 
contractor equivalents that support the DoD acquisition workforce, and include the 
estimates as supplementary DoD reporting data to Congress. If contractor equivalent 
estimates are not included, the total efforts and needs of the Refined Packard workload 
will not be recognized. 


Management Comments on the Finding and Audit Response 


Management Comments on DMDC Role in Acqnisition Workforce Counts. The 
Director, DMDC stated the report inaccurately reflected the DMDC role in providing 
refined Packard counts. The Director stated the DMDC primarily receives and compiles 
data from the Military Services and Office of Secretary of Defense agencies, which are 
allowed to modify workforce counts without justification after an initial DMDC analysis. 
The Director stated that DMDC was not in a position to provide reconciliations on 
workforce counts or provide written documentation on significant increases or decreases 
in DoD acquisition workforce personnel. The Director, DMDC deferred to the role and 
responsibility of the Director, Defense Procurement and Acquisition Policy in developing 
and implementing standard procedures and guidance. 

Audit Response. DoD Instruction 5000.55, “Reporting Management Information on 
DoD Military and Civilian Acquisition Personnel and Positions,” November 1, 1991, 
designates DMDC to serve as custodian of all automated records collected and to provide 
data quality control, inquiry capabilities, and administrative and computer support. A 
May 13, 1999, Under Secretary of Defense (AT&L) policy memorandum, emphasizes 
that the DMDC database is the official repository for acquisition workforce personnel 
counts. We agree that the Director, Defense Procurement and Acquisition Policy is 
responsible for developing and implementing acquisition workforce standard procedures 
and guidance; however, we believe that the existing DoD policy requires DMDC to 
provide accurate and reliable acquisition workforce information. Without DMDC 
measures to ensure the accuracy of periodic workforce submissions, DMDC role would 
be limited to compiling and maintaining unreliable data. 


Recommendations, Management Comments, and Audit Response 


Renumbered and Redirected Recommendations. As a result of Office of 
USD(AT&L) comments we renumbered draft report Recommendation A.l ,b.(2) as 
Recommendation A.3. in the final report and redirected it to the Under Secretary of 
Defense for Personnel and Readiness to comment in coordination with the USD(AT&L). 
We renumbered draft report Recommendation A.l.b(l) as final report 
Recommendation A. 1 .b. 
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A.I. We recommend that the Director, Defense Procurement and Acquisition 
Policy: 


a. Develop and implement written standard operating procedures and 
guidance for counting the acquisition workforce to include: 

(1) a definition of the acquisition workforce count that includes the 
Acquisition Organization Count and the Refined Packard workforce count. 

(2) a definition of the methodologies and procedures used to perform 
periodic workforce counts derived from Military Services and Fourth Estate 
databases. 


(3) requirements to maintain and support acquisition workforce count 
documentation as required by DoD Instruction 5000.55. 

b. Revise DoD Instruction 5000.55 to update the information requirements 
for automated data files to ensure consistent acquisition workforce information 
format and reporting from the DoD components. 

Management Comments. The Director, Human Capital Initiatives, Office of the 
USD(AT&L), responding for the Director, Defense Procurement and Acquisition Policy, 
concurred wto Recommendations A. 1 .a. and A. 1 .b., stating that he directed a complete 
update of the AT&L workforce management information requirements, which included a 
review and update of the workforce count definition. The Director stated that 
USD(AT&L) will ensure supporting workforce count methodology and procedures are in 
place and that requirements are in place to ensure maintenance and support of workforce 
count documentation. The Director also stated that standard policy, guidance, and 
process updates are being implemented with a new count methodology to facilitate 
workforce analysis and human capital strategic planning. 

Audit Response. Although the Director, Human Capital Initiatives, concurred with 
Recommendation A.I., the comments did not address the time line for implementation of 
the written standard operating procedures and guidance and revision to the DoD 
Instruction. In separate follow-up correspondence the Office of the USD(AT&L) noted 
that the implementation would be by August 31, 2006. 

A.2. We recommend that the Director, Defense Manpower Data Center develop a 
knowledge management program to maintain corporate knowledge of Defense 
Manpower Data Center information systems and processes. 

Management Comments. The Director, DMDC concurred with the recommendation, 
stating that DMDC did not fully pass along knowledge related to the methods used in 
attempting to identify personnel in the acquisition workforce. The Director stated that 
the DMDC has taken steps to rectify the situation. 

Audit Response. Although the Director, DMDC concurred with Recommendation A.2., 
the comments were not fully responsive because they did not address the development of 
a knowledge management program or identify specific steps taken to pass along 
knowledge and did not provide a timetable to implement such a program. We request 
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further comments from the Director, DMDC regarding implementation of 
Recommendation A. 2. 

USD(AT&L) Comments. Although not required to comment, the Director, Human 
Capital Initiatives, Office of the USD(AT&L), commented that his office would work 
closely with the Under Secretary of Defense for Personnel and Readiness to clarify 
requirements, policy, and processes that impact the management information services 
provided through DMDC. 

A.3. We recommend that the Under Secretary of Defense for Personnel and 
Readiness in coordination with the Office of the Under Secretary of Defense for 
Acquisition, Technology, and Logistics revise DoD Instruction 5000.55 to estimate 
contractor equivalents, to the best extent estimable, who support the DoD 
acquisition workforce, and include such estimates as supplementary acquisition 
workforce reporting data in annual DoD reporting to Congress. 

USD(AT&L) Comments. The Director, Human Capital Initiatives, Office of the 
USD(AT&L), commenting to the draft report recommendation directed to the Director, 
Defense Procurement and Acquisition Policy, stated the recommendation came under the 
purview of the office of the Under Secretary of Defense for Personnel and Readiness 
because that office had broader authority to establish the requirement and direct 
collection of DoD contractor personnel data. 

Audit Response. As a result of Office of USD(AT&L) comments, we have renumbered 
and redirected the recommendation in the final report to the Under Secretary of Defense 
for Personnel and Readiness to respond in coordination with the Office of the 
USD(AT&L). 

Air Force Comments. Although not required to comment, the Military Deputy, Office 
of the Assistant Secretary of the Air Force (Acquisition) stated there is currently no 
requirement for DoD to report acquisition workforce contractor equivalents and 
questioned the relevancy of counting contractors. 
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B. Contracted Support Services for the 
Acquisition, Technology, and 
Logistics Workforce 

The Defense Contracting Command-Washington (DCC-W) did not 
properly negotiate or administer support service contract 
DASW01-03-F-0393 to Jefferson Solutions. In addition, the contracting 
officer’s representative (COR) approved Jefferson Solutions’ monthly 
invoices without requesting written support for services performed, hours 
expended, or products provided. The contract negotiation and 
administration problems occurred because: 

• DCC-W did not maintain support for the independent 
Government cost estimate (IGCE) determination of labor 
hours, labor rates, and direct and indirect costs, which appeared 
to be based on unverified contractor proposal data; 

• DCC-W did not require signatures on or dating of the IGCE 
and did not maintain letters of delegation to the COR; 

• DoD contracting officials did not place adequate emphasis on 
contract administration and surveillance; and 

• DCC-W did not require written monthly progress reports. 

As a result, the Government was not assured that it negotiated fair and 
reasonable prices with Jefferson Solutions, that work performed by 
Jefferson Solutions in identifying and reporting the Refined Packard 
workforce count was done properly for the time and materials expended, 
nor that Government resources were used economically. 


Refined Packard Workforce Count Support Contracts 


DCC-W, on behalf of the Office of the USD(AT&L), awarded the DoD-wide 
Refined Packard workforce identification contracts. Since 1997, DCC-W has 
awarded five contracts to Jefferson Solutions to support DoD acquisition 
workforce issues. DCC-W and Jefferson Solutions had no records relating to the 
initial contract. The subsequent four contracts are valued at $2,447 million. The 
following table summarizes the last four contracts awarded. 


17 



154 


Table 5. Contract Actions Reviewed 

Contract No. 

Award Date 

Contract Value 

DASW01-99-F-1028 

March 15, 1999 

$451,194 

DASWOl-Ol-F-0349 

December 18, 2000 

$413,906 

DASW01-02-F-0538 

February 1, 2002 

$432,679 

DASW01-03-F-0393 

March 1, 2003 

$1,149,259 

Total 


$2,447,038 


Prior Contracts. On March 15, 1999, DCC-W awarded a firm-fixed-price 
contract (DASW01-99-F-1028) for a period of 10 months to Jefferson Solutions, 
Modification POOOOl extended the period of performance to January 14, 2001, for 
the continued implementation of the Acquisition and Technology workforce 
identification. Jefferson Solutions was to provide to the Office of the 
USD(AT&L) both consulting expertise and advice for further refinement of the 
Refined Packard methodology. The total cost of the contract was $451 ,194. The 
contracting officer could not locate a 1997 contract awarded to Jefferson 
Solutions for reporting the acquisition workforce and stated the contract was sent 
to a warehouse and may have been destroyed. 

On December 18, 2000, DCC-W awarded a firm-fixed-price contract 
(DASWOl-Ol-F-0349) for a period of 12 months to Jefferson Solutions to provide 
consulting expertise and advice relevant to further refinement of the Refined 
Packard algorithm. Modification POOOOl extended the period of performance to 
January 31, 2002. The contract was not competitively awarded and was follow- 
on work to contract DASW01-99-F-1028. Total cost of the contract was 
$413,906. 

On February 1, 2002, DCC-W awarded a firm-fixed-price contract 
(DASW01-02-F-0538) for a period of 12 months to Jefferson Solutions to provide 
for the full identification and quantification of the DoD key acquisition workforce 
based on the Refined Packard methodology. Modification POOOOl extended the 
contract one month to February 28, 2003. The total cost of the contract was 
$432,679. The contract was not competitively awarded and was follow-on work 
to contract DASWOl-Ol-F-0349. A sole-source justification stated that Jefferson 
Solutions was the only contractor that could continue the effort in a timely and 
cost-effective fashion. The sole-source justification was not signed or dated. 

Current Contract. On March 1, 2003, DCC-W awarded a firm-fixed-price 
contract (DASW01-03-F-0393) for a base period of 1 1 months and up to 3 option 
years to Jefferson Solutions to provide for the full identification and 
quantification of the DoD-wide acquisition workforce, based on the methodology 
refined by the December 1997 DoD Workforce Identification Working Group. 
The base year was awarded for $266,699. As of March 14, 2005 (modification 
P00006), the total cost of the contract was $1,149,259. 
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Government Oversight of Refined Packard Workforce Count 
Contractor 


Based on a review of support service contract DASW01-03-F-0393: 

• DCC-W did not properly negotiate or administer support service 
contract DASW01-03-F-0393 to Jefferson Solutions. A DCC-W 
award to Jefferson Solutions was based on an unsupported IGCE. The 
DCC-W contracting officer should have performed a price analysis to 
ensure the Government received a fair and reasonable price. The 
contractor was not required to provide written monthly progress 
reporting. 

• The COR approved 28 Jefferson Solutions monthly invoices for 
contract DASW01-03-F-0393 from March 2003 through June 2005 
without requesting written support for services performed^ hours 
expended, or products provided. Each of the contractor invoices stated 
only “Provide for the full identification and quantification of the DoD- 
wide Acquisition Technology and Logistics (AT&L) Workforce, 
based on the methodology refined by the December 1997 DoD 
Workforce Identification Working Group.” None of the monthly 
invoices provided any further information on the monthly activities 
performed to support payment. 


Contract Negotiation and Administration 


The contract negotiation and administration problems occurred because DCC-W 
did not maintain support for the IGCE determination of estimated amounts and 
did not require signatures on or dating of the IGCE. DCC-W did not maintain 
letters of delegation to the COR and did not place adequate emphasis on contract 
administration and surveillance of Jefferson Solutions products. 

Independent Government Cost Estimate. Federal Acquisition Regulation 
Subparts 15. 404- 1(b)(2) and 15. 404- 1(c)(2) note the Government may use various 
cost or price analysis techniques to determine a fair and reasonable price. Such 
techniques include a comparison of costs proposed by the offeror for individual 
cost elements with an independently prepared Government estimate of the costs. 
DCC-W could not support its estimated amounts for labor hours, labor rates, and 
direct and indirect costs for contract DASW01-03-F-0393. The IGCE consisted 
of a list of labor categories, rates, and hours. DCC-W maintained no explanation 
of the sources or analyses pertaining to the estimated cost information. The IGCE 
appeared to be based on data from the contractor proposal and not from a price 
analysis completed by the Government to ensure the Government received a fair 
and reasonable price. The IGCE total estimated cost and fee was $266,860. The 
total firm-fixed price for the base year for the contractor proposal was $266,699. 
The difference between the IGCE and the contractor price proposal was only 
$161. In addition, the IGCE did not include an estimate for the three option 
periods in the proposal. The proposed first two option periods and prices were 
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included in the negotiated contract without change. Further, the contracting 
officer stated that DCC-W policy did not require Government signatures on and 
dating of IGCEs or estimates of contractor-proposed option periods until June 
2005. 

Fair and Reasonable Price. The Memorandum of Negotiations for contract 
DASW01-03-F-0393 notes that a request for quotes was submitted to 
10 contractors, using the GSA Advantage E-Buy Web site. Jefferson Solutions 
was the only contractor that submitted a proposal. The memorandum stated that 
the COR reviewed the Jefferson Solutions price proposal and determined the costs 
were realistic for the work to be performed when compared with previous 
contracts for the same type of work. All previous contracts were awarded to and 
performed by Jefferson Solutions. DCC-W contracting officials should have 
performed a price analysis to ensure the Government received a fair and 
reasonable price. 

The memorandum also noted that because the award was based on full and open 
competition, a detailed price analysis was not performed in accordance with the 
requirements of Federal Acquisition Regulation Subpart 15.804-3(b). The 
memorandum’s reference to Subpart 15.804-3(b) is erroneous because the subpart 
did not exist at the time of the February 3, 2003, solicitation.* The memorandum 
did not reference Federal Acquisition Regulation Subpart 1 5.404-l(a)(2) 
requirement that a price analysis shall be used when cost or pricing data are not 
required. Had the contracting officer required a price analysis, he could have 
demonstrated whether the proposed price was reasonable in comparison with 
current or recent prices for the same or similar items. 

COR Letter of Delegation. The Defense Federal Acquisition Regulation 
Supplement Subpart 201 .602-2 and the DCC-W “Contracting Officer’s 
Representative (COR) Guide,” (COR Guide) April 15, 2005, note the 
appointment of a COR must be made by the contracting officer in writing. 

Subpart 201 .602-2 and the DCC-W COR Guide additionally require that the 
designation specify the extent of the COR authority to act on behalf of the 
contracting officer, identify the limitations on the COR authority, specify the 
period covered by the designation, state the authority is not redelegable, and state 
that the COR may be personally liable for unauthorized acts. 

While the Memorandum of Negotiations for contract DASW01-03-F-0393 
referred to COR actions, the DCC-W contracting officer did not designate a COR 
in writing as required by the Defense Federal Acquisition Regulation Supplement. 
The contracting officer stated that DCC-W does not use the term COR for GSA 
Federal Supply Schedule contracts; rather, it uses the term “point of contacf ’ to 
refer to the COR function. DASW01-03-F-0393 and prior DCC-W contracts with 
Jefferson Solutions designated points of contact in the offices of the USD(AT&L) 
or the Director, Defense Procurement and Policy. However, the designations did 
not meet the requirements of Defense Federal Acquisition Regulation Supplement 
Subpart 201.602-2 and the DCC-W COR Guide because they did not specify the 


® Review of prior Federal Acquisition Regulation revisions indicated that no Federal Acquisition 
Regulation 15.804 reference existed past July 1997. 


20 



157 


period covered by the designation, the extent or limitations of the COR authority, 
or state the authority was not redelegable. 

Contract Administration and Surveillance and Contractor Progress 
Reporting. The DCC-W COR Guide noted the COR was required to maintain 
effective surveillance of the contract and document the surveillance performed. 
Surveillance documentation includes progress reports submitted by the contractor. 
COR/point of contact surveillance files were limited and consisted of a copy of 
the contract, e-mails, contractor invoices, and COR/point of contact approvals. 
COR/point of contact surveillance consisted primarily of a review of contractor 
invoices, e-mails, and verbal progress reports over the telephone with the 
contractor. The contracting officer could not explain why the contract did not 
require written monthly progress reports, and noted the COR/point of contact 
could not properly monitor contractor performance to determine whether work 
was properly performed and progress was being made without written monthly 
progress reports. 

The Office of the Defense Procurement and Policy COR/point of contact noted 
after our review of their contracting files that the contractor should be required to 
provide written monthly progress reports. On October 21, 2005, the contracting 
officer issued modification P00007 to contract DASWOl -03-F-0393 to require the 
contractor to provide USD(AT&L) with written monthly progress reports. 


Summary 

A DCC-W award to Jefferson Solutions was justified through an unsigned, 
undated, and unsupported IGCE. The Government was not assured that fair and 
reasonable prices were negotiated because the IGCE lacked sufficient detail for 
supporting labor rates, labor categories, labor hours, and direct as well as indirect 
costs, and no one performed a price analysis. DCC-W contracting officials should 
have performed a price analysis to ensure the Government received a fair and 
reasonable price. The contracting officer took corrective action to require the 
contractor to provide written monthly progress reports. 
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Recommendation and Management Comments 


B.l. We recommend that the Commander, Defense Contracting Command- 
Washington require that acquisition workforce count contracts supporting 
Office of Under Secretary of Defense for Acquisition, Technology, and 
Logistics components include written contractor monthly progress reports to 
the contracting officer and points of contact or contracting officer 
representatives. 

Army Comments. The Director, Army Contracting Agency, responding for the 
Director, Defense Contracting Command-Washington, concurred with the 
recommendation and noted that any future contract requirements for these or 
similar services will include the proper Government and contractor reporting 
requirements. 

Office Under Secretary Defense for Acquisition, Technology & Logistics 
Comments. Although not required to comment, the Director, Human Capital 
Initiatives, Office of the Under Secretary of Defense for Acquisition, Technology 
and Logistics, commented that the final option for the workforce count support 
contract was not exercised and that a new acquisition is planned. 
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Appendix A. Scope and Methodology 


We performed this audit from May 2005 through January 2006 in accordance 
with generally accepted government auditing standards. The audit analyzed 
existing Federal, DoD, and Service (including DLA) acquisition workforce policy 
and guidance, assessed Service and DLA compliance with acquisition workforce 
guidance regarding workforce counts, and evaluated management controls over 
the acquisition workforce counting process. This analysis included determining 
the processes used by the Office of the Secretary of Defense to count and report 
the acquisition workforce and determining the extent of support for counts made 
through the Refined Packard method. The audit did not determine the extent of 
support for workforce counts made through the Acquisition Organization method 
as those counts were derived directly from DoD civilian and military personnel 
databases. We reviewed documents from FY 1997 through FY 2005. 

In addition, the audit team held focus group discussions with management and 
nonmanagement acquisition workforce personnel, including program 
management, contracting, quality assurance, and engineering work groups, to 
identify areas of concern to the workforce and prepare a potential DoD-wide 
questionnaire on acquisition workforce issues for use in later audits. 

The audit team conducted site visits to Office of the Secretary of Defense 
components including the Office of the USD(AT&L) and its Defense 
Procurement and Acquisition Policy and Defense Acquisition University 
components, the Office of the Under Secretary of Defense for Personnel and 
Readiness and its DMDC component, and support contractor Jefferson Solutions. 
The audit also performed site visits or met with personnel from the Army 
Acquisition Support Center; the Army Contracting Agency; the Office of the 
Assistant Secretary of the Navy for Research, Development, and Acquisition; the 
Naval Sea Systems Command; the Office of the Assistant Secretary of the Air 
Force for Acquisition; the Air Force Materiel Command; the Aeronautical 
Systems Center; the Air Force Space and Missile Systems Center; DLA 
Headquarters; Defense Supply Center-Columbus; and Defense Supply Center- 
Richmond. We determined the use and extent of contracted support services at 
acquisition workforce field organizations and locations visited. 

Use of Computer-Processed Data. We relied on data received from the Militaiy 
Services, DMDC, and DLA information systems for the FY 2004 Refined 
Packard workforce counts. We did not test those information systems. Issues that 
came to our attention are included in Finding A. 

Government Accountability Office High-Risk Area. The Government 
Accountability Office has identified several high-risk areas in DoD. This report 
provides coverage of the Strategic Human Capital Management and the DoD 
Approach to Business Transformation— Weapon Systems Acquisition high-risk 
areas. 
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Appendix B. Prior Coverage 


During the last 5 years, the Government Accountability Office (GAO) and the 
Department of Defense Inspector General (DoD IG) have issued seven reports 
discussing the DoD acquisition workforce. Unrestricted GAO reports can be 
accessed over the Internet at http ://ww w, gao . gov . Unrestricted DoD IG reports 
can be accessed at httPi/Avww.dodig.inil/audit/reports . 


GAO 


GAO Report No. GAO-05-233, “Federal Acquisition: Progress in Implementing 
the Services Acquisition Reform Act of 2003,” February 2005 

GAO Report No. GAO-04-83, “Human Capital: Implementing Pay for 
Performance at Selected Personnel Demonstration Projects,” January 2004 

GAO Report No. GAO-03-443, “Federal Procurement: Spending and Workforce 
Trends,” April 2003 

GAO Report No. GAO-03-55, “Acquisition Workforce: Status of Agency Efforts 
to Address Future Needs,” December 2002 

GAO Report No. GAO-02-737, “Acquisition Workforce: Agencies Need to Better 
Define and Track the Training of Their Employees,” July 2002 

GAO Report No. GAO-02-630, “Acquisition Workforce: Department of 
Defense’s Plans to Address Workforce Size and Structure Challenges,” April 
2002 


DoD IG 

DoD IG Report No. D-200 1-129, “Contracting Officer Determinations of Price 
Reasonableness When Cost or Pricing Data Were Not Obtained,” May 30, 2001 
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Appendix C. DoD Acquisition Organizations List 


DoD Instruction 5000.55, November 1, 1991, defines an acquisition organization 
as one of 22 listed organizations and any successor organization of these 
commands, including subordinate elements, whose mission includes planning, 
managing, and executing acquisition programs. The list is used as the basis for 
the Acquisition Organization workforce counting method and is shown below: 

Office of the Secretary of Defense 

Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics 


Department of the Army 

Army Acquisition Executive 

Army Materiel Command 

Army Space and Missile Defense Command 


Department of the Navy 

Office of the Assistant Secretary of the Navy for Research, Development, and 
Acquisition 

Naval Air Systems Command 
Naval Sea Systems Command 
Naval Supply Systems Command 
Naval Facilities Engineering Command 
Naval Space and Warfare Systems Command 
Naval Strategic Systems Program Office 
Office of Naval Research 

Navy Program Executive Office/Direct Reporting Program Manager Organization 
Marine Corps Systems Command 


Department of the Air Force 

Office of the Assistant Secretary of the Air Force for Acquisition 

Air Force Materiel Command 

Air Force Space and Missile Center 

Air Force Program Executive Office 


Combatant Command 

U.S. Special Operations Command-Acquisition Center 
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Other Defense Organizations 

Defense Logistics Agency 

Defense Contract Management Agency 

Missile Defense Agency 
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Appendix D. Military Services and Defense 
Logistics Agency Acquisition 
Workforce Counting Processes 

The FY 2004 and prior Refined Packard workforce counts were unverifiable and 
may not accurately reflect the true DoD acquisition workforce. Guidance and 
processes used by the services to count and report the acquisition workforce vary 
greatly. 


Army Counting Process. The Military Deputy to the Army’s Acquisition 
Executive serves as the Army DACM. The Director, Army Acquisition Support 
Center serves as the Army Deputy Director Acquisition Career Management. 

The Army DACM reports quarterly a Refined Packard acquisition workforce 
count to DMDC as required by DoD Instruction 5000.55. The Army’s Career 
Acquisition Personnel and Position Management Information System (Army 
information system) is the Army database system for Army’s acquisition 
workforce and is used to report data to DMDC. The Army Acquisition Support 
Center uses the Army information system for acquisition workforce training, 
personnel history, and qualifications. Army personnel noted that DMDC can also 
obtain Army-related acquisition workforce information directly from the Defense 
Civilian Personnel Data System. The Army’s information system comprises 
acquisition workforce data from multiple sources including the Defense Civilian 
Personnel Data System, the Total Officer Management Information System, and 
manually counted data of enlisted personnel. 

Revisions to Army Workforce Counting Methodology. Army 
Acquisition Support Center officials staled that while the Refined Packard 
workforce count provides a good definition of the acquisition workforce, the 
Army may expand its acquisition workforce count by at least 1 2,000 personnel 
(25 percent) to include facilities engineering personnel at three major Army 
commands, including the Corps of Engineers-Civil Works, the Installation 
Management Agency, and Office of the Assistant Chief of Staff for Installation 
Management. Army Acquisition Support Center personnel stated the expansion 
would begin with the FY 2006 Refined Packard workforce count as a result of 
personnel description changes made during FY 2005. 

Army Workforce Projection Methods. The Office of the Assistant 
Secretary of the Army for Acquisition, Logistics, and Technology is responsible 
for sizing the Army acquisition workforce. The Army uses a Forecast Framework 
Model to assess the current workforce count, project future workforce size, and 
conduct a “gap analysis’’ comparing current demographics with the desired 
demographics and flagging significant differences. The acquisition workforce 
projections are analyzed by the Army Acquisition, Logistics, and Technology 
personnel to identify any significant growth in accession requirements. The 
analysis identifies Army acquisition workforce reductions due to retirements or 
migrations or where a new or increased workload requirement has been projected. 
The Army also considers industry and academia information describing emerging 
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technical skills and acquisition community best practices in strategic planning 
recommendations. 

Navy Counting Process. The Department of Navy DACM has the primary 
responsibility of developing and implementing policies and procedures for the 
Navy Refined Packard workforce count and the Navy AT&L Workforce 
Education, Training, and Career Development Program on behalf of the Assistant 
Secretary of the Navy for Research, Development, and Acquisition, who serves as 
the Navy Service Acquisition Executive. The Navy DACM is also responsible 
for establishing and maintaining a management information system to support 
workforce management, compliance monitoring, tracking, and reporting for the 
Department of the Navy Acquisition Workforce. 

Navy Workforce Counting Systems. The Navy DACM relies on civilian 
and military acquisition personnel and position data systems that are updated 
monthly by Department of the Navy command levels. The Navy DACM does 
not use the Refined Packard method to assemble the workforce count. Instead, 
the Navy DACM uses the Refined Packard methodology as a tool to assist the 
components in identifying and assimilating positions into the acquisition 
workforce. The Navy DACM uses a Navy computer program to pull coded 
Department of the Navy acquisition workforce positions from the Defense 
Civilian Personnel Data System, the Navy Total Force Manpower Management 
System, the Navy Officer Personnel Information System, and the Marine Coips 
Acquisition Workforce System of Management. The acquisition positions within 
the personnel and position data systems are coded by management level personnel 
at the Department of Navy commands. The Navy DACM process removes all 
nonacquisition personnel, including some personnel considered as Refined 
Packard Method Category One. However, the Navy DACM stated that DMDC 
and Jefferson Solutions revised the Navy DACM submission for the FY 2004 
DoD-wide acquisition workforce count by adding back approximately 
200 Category One personnel that had been eliminated by this Navy DACM 
process. 

Navy Workforce Counting Criteria. Secretary of the Navy 
Instruction 5300.36, “Department of the Navy Acquisition Workforce Program,” 
May 31, 1995, and the Navy DACM, “Interim Guidance Department of the Navy 
(DoN) Acquisition, Technology, and Logistics (AT&L) Workforce Education, 
Training, and Career Development Program,” October 11, 2005, are supplied to 
major commands as guidance for implementing acquisition workforce coding and 
revised Defense Acquisition Workforce Improvement Act requirements. The 
Navy DACM noted that the interim guidance is consistent with the revised DoD 
Directive 5000.52 and should replace the same topics covered in Secretary of the 
Navy Instruction 5300.36. The Navy DACM also stated that the Navy will 
publish a revised Secretary of the Navy policy instruction and operating guidance 
by July 2006, after the Office of the USD(AT&L) publishes a DoD operational 
instruction and Web-based Desk Guide. 

Neither the Secretary of the Navy Instmction nor the Navy interim guidance 
includes procedures for the Navy DACM or component of the Department of the 
Navy commands to verify the validity of coded acquisition workforce data. The 
Navy DACM stated that its office did not maintain written standard operating 
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procedures for assembling the count or to verify the validity of coded workforce 
data made by the Department of the Navy commands. However, the Navy 
DACM noted that its office had requested the Naval Audit Service to validate the 
data through audit examination. 

Air Force Counting Process. The Air Force DACM leads the Acquisition 
Professional Development Program on behalf of the Service Acquisition 
Executive. The Air Force DACM is responsible for the Air Force acquisition 
workforce count submitted to DMDC. 

Air Force Workforce Counting Systems. The Air Force DACM staff 
performs an acquisition workforce count by retrieving data on acquisition coded 
positions from four Air Force systems: a DoD Instruction 5000.55 file, an 
Acquisition Career Management System, the Air Force Personnel Data System,* 
and the DMDC Refined Packard workforce count. The Air Force DACM staff 
includes in the Air Force acquisition workforce all personnel whose social 
security numbers are in at least three of the four systems or if they were in both 
the Acquisition Career Management System and the Air Force Personnel Data 
System. The Air Force DACM staff submits information to DMDC to allow 
DMDC to make an assessment of the Air Force acquisition workforce count. 
DMDC sends results back to the Air Force DACM staff for review and 
correction. The Air Force DACM staff sends corrections back to DMDC in a 
spreadsheet format. 

Air Force Workforce Counting Criteria, An undated Web-based Air 
Force acquisition guidance implements Air Force Major Command coding of 
acquisition positions in the manpower and data personnel systems. The Air Force 
coding guidance notes that when job duties are predominantly acquisition-related, 
then the position falls under the requirements of the Air Force’s Acquisition 
Professional Development Program. The Air Force DACM staff noted there were 
data entry and coding problems with personnel entering the data at Air Force 
bases. The Air Force DACM staff was in the process of writing new policies and 
procedures for coding and counting the acquisition workforce. The new policies 
and procedures were expected to be completed for the FY 2006 end-of-year 
acquisition workforce count. 

DLA Counting Process. The DLA Customer Support Office in Columbus, Ohio, 
submits quarterly DLA Acquisition Workforce count data directly to the Defense 
Manpower Data Center. DLA has a Component Acquisition Executive 
responsible for all acquisition functions within the agency. The Component 
Acquisition Executive serves on the DLA Acquisition Career Program Board, 
which periodically convenes to discuss and make decisions on acquisition 
workforce policy issues affecting DLA. However, DLA is not required by DoD 
guidance to maintain a DACM to assist in DLA acquisition workforce training, 
education, and career development. The Defense Acquisition University’s 
Defense Agency Workforce Support Office performs DACM functions in support 
of the Defense agencies. 


* The Air Force Personnel Data System includes data from the DoD Military Personnel Data System and 
the Defense Civilian Personnel Data System 
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DLA Workforce Counting Systems. The Customer Support Office 
produces quarterly acquisition workforce counts through extraction of DLA 
Acquisition Personnel File records from the Defense Civilian Personnel Data 
System and use of DLA Acquisition Position Files. The Acquisition Personnel 
File and Acquisition Position File records are periodically updated by the DLA 
Customer Support Offices in Columbus, Ohio, and New Cumberland, 
Pennsylvania. DMDC performs a match between the Acquisition Personnel File 
records and Acquisition Position File records; the results of that match become 
the DLA acquisition workforce count. 

In FY 2004, the Customer Support Office also provided DMDC the acquisition 
workforce count data for the Defense Contract Management Agency. 

Compilation of the Defense Contract Management Agency acquisition workforce 
count was transferred to the Department of the Army in April 2005. Annually, 
Jefferson Solutions asks the DLA Headquarters Human Resource Policy and 
Information Office to review and comment on the Refined Packard workforce 
count for that year. For the FY 2004 count, DLA did not provide comments to 
Jefferson Solutions. 

DLA Workforce Counting Criteria. DLA uses the DoD 
Instruction 5000.55 as guidance and does not maintain supplementary criteria 
regarding maintaining and reporting acquisition workforce data. Once a planned 
revised DoD instruction is issued, DLA personnel stated that the agency will 
review and determine if any supplementary internal guidance is necessary. The 
Customer Support Office does not maintain standard operating procedures but 
does maintain standard reporting queries and schedules for the DLA acquisition 
workforce count. 
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Appendix E. Report Distribution 


Office of the Secretary of Defense 

Under Secretary of Defense for Acquisition, Technology, and Logistics 
Director, Defense Procurement and Acquisition Policy 
Director, Acquisition Resources and Analysis 
Under Secretary of Defense for Personnel and Readiness 
Director, Defense Manpower Data Center 
Under Secretary of Defense (Comptroller)/Chief Financial Officer 
Deputy Chief Financial Officer 
Deputy Comptroller (Program/Budget) 

Director, Program Analysis and Evaluation 


Department of the Army 

Office of the Administrative Assistant to the Secretary of the Army 
Commander, Defense Contracting Command-Washington 
Auditor General, U.S. Army Audit Agency 


Department of the Navy 

Naval Inspector General 

Auditor General, Naval Audit Service 


Department of the Air Force 

Assistant Secretary of the Air Force for Financial Management and Comptroller 
Auditor General, Department of the Air Force 


Combatant Command 

Inspector General, U.S. Joint Forces Command 


Other Defense Organization 

Director, Defense Logistics Agency 
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Non-Defense Federal Organizations and Individuals 

Office of Management and Budget 


Congressional Committees and Subcommittees, Chairman and 
Ranking Minority Member 

Senate Committee on Appropriations 

Senate Subcommittee on Defense, Committee on Appropriations 
Senate Committee on Armed Services 

Senate Committee on Homeland Security and Governmental Affairs 
House Committee on Appropriations 

House Subcommittee on Defense, Committee on Appropriations 
House Committee on Armed Services 
House Committee on Government Reform 

House Subcommittee on Government Management, Finance, and Accountability, 
Committee on Government Reform 

House Subcommittee on National Security, Emerging Threats, and International 
Relations, Committee on Government Reform 
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Under Secretary of Defense for Acquisition, 
Technology, and Logistics Comments 



TemN^SsY 

iMirncc 


OFFICE OF THE UNDER SECRETARY OF DEFENSE 
3000 DET6NS6 PENTAGON 
WASHINGTON. DC 2030( 3000 


MAR 142008 


MEMORANDUM FOR PROGRAM DIRECTOR, CONTRACT MANAGEMENT. 

OFFICE OF INSPECTOR GENERAL, DEPAR'I'MENTOF 
DF-FENSE 


THROUGH DIRECTOR, ACQUISITION RESOURCES AND ANALYSIS 



SUBJECT: Draft Audit Report, “Report on the DoD Acquisition Workforce Count,” 
(Project No. D2005-DO0OCB-0181) 


This is in response to ywr February 13. 2006 memorandum asking for our 
comments on subject draft report, Our response to each of the audit repent 
rccommerdauons is attached. 

We ajqjrcciate the opportunity to rc^ond to your draft audit repon and look 
forward to resolving the cited itsues. My point of contact is Mr. Gariy Shafovabff, He 
can be contacted by telephone at (703) 805-4969 or email at garrv .sha fovalQf!i@dau .mii • 


Attachment: 
As stated 


cc: 

OUSD (AT&L) 

OUSD (P&R)/DMDC 
OUSD (AT&Ll/DPAP 
DOA/DCCW 

OSD AT&L Workforce Management Group Members 


0 
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Din»ior, Human Capital Initiatives (OUSD (AT&L)) 
Comments on Draft DoD 10 Audit Report 
"Report on the DoD Acquisiliem Woikforcc Count" 
OIG Project No. D2005-DO00CD-0181 


QIC ReconimgfKlation A.l.a.fn. Der-elop and implement written standard operating 
procedures and guidance for counting the acquistlion workforce to include: (1) a 

definition of the acquisiliwi workforce count which includes the Acquisition \ 

Organization Count and die Refined Fadcard Count. j 

OUSD fAT&Lt Resnrmse Concur. The Director, Human Capital Initiatives 

(OUSD (AT&L)) has recently directed a complete update of AT&L workforce 
management inforraalioo TcqulTemcots, to include a review and update of the workforce 
count definition. As pan of this effort the workforce count definition and supportbg 
processes ore being updated. 

QIC Recommendation .A.l.a.(21 . Develop and implement written standard operating 
procedures and guidance for counting the acquisitton workforce to include; (2) a 
definition of the methodologies and procedures used to perfonn periodic uxirkforcc 
counts derived from Military Services and Fourth Estate databases. 

ni ISP (AT&f t R*tnot>«s A.l.ai2V Concur. OUSD (AT&L) is updating the count 
definition and will ensure trupporting methodology and procedures are in place. 

QIC Recommendatton Develop and implcitient written standard operating 

procedures and guidance for counting (he acquisition wwkforce to include: (3) 
requirements to maintain and support acquisition workforce count documentation as 
required by DoD Instruction 5000.55. 

OUSD aT&i-l Response A.1.a.(31. Concur. OUSD (AT&L) will ensure appropriate 
requirements are in place to ensure maintenance and support of workforce count 
documentation. 

Olfi Recommendarlon A.l.b.dl . Revise DoD Instruction 5000.55 to update dte 
inforroation lequircments for ttulomatcd data files to ensure consistent acquisition 
workforce information formal and reporting from the DoD components. 

OUSD (AT&L) RcspPBse A.I.bJIk Concur. Management information requirements 
are being updated. Standard policy, guidance and process updates arc being 
implemented with a new count methodology to facilitate contprehensive workforce 
analysts and human capital .strategic planning. 
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Final Report 
Reference 


OtG Recommendation A.l.b.f2t . l-stimaie contraclor equivalents, w the best extent 
estimable, which support the E>oD acquisition workforce, and include such esliraales as 
supplementary acquisition workforce reporting data in annual DoD rqjotling to 
Congress. 

OUSD fATAT.l Hemnnse A.l.b.f2) . This pfo|K«al would have to be worked by OUSD 
(P&R) to address the broader issue of reporting for DoD contractor personnel. OSD 
(P&R) is the proper authority for e.stablishing the requirement and directing collection. 

01G Recommendation A.2 . We recommend the Director, Defense Manpower Dais 
Center (DMDC) develop a knowledge management program us maintain corporate 
knowledge of Defense Manpower Data Center information systems and processes. 

QUSD fAT&U Response A.2 . the Director. Human Capital Initiatives (OUSD 
(AT&1-) will work closely with OUSD (PiKVCPMS to ensure clarity of requiremenu. 
policy and prtwesses which impact the management information services provided by 
through DMDC and the Defense Civilian Personnel Daw System (DCPDS). 

OIG Recommendation B.l . We recoromcml that the Commander, Defense Contracting 
Command-Washingtoo require that acquisition workforce count contracts supporting 
Onicc of Under Secretary of Defense for AT&L components include written contractor 
monthly progresa reports to the contracting officer and pointt of contact or contracting 
officer tqvrescntatives. 

OUSD Response B.l . The final option of the workforce count contractor 

support contract was not exercised. A new acquisition is planned for contractor support 
for (he AT’&L Kunwn Capital fnitiaiives/workforcconice. 


Renumbered 

A.3 

Redirected 


Renumbered 

A.3 

Redirected 
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Department of the Army Comments 



PEPARTMBNT OF THE ARMY 

Ohuct oune sec«iciAfiTO^r»<feA»«»T 

•comsmon toots ncs •»«> iechnouioy 
ua UM1 COtRRACTMQ aCCHCY 
sio» LEESBtAG PMC sune 102 

FU i » CHUMtt V*> 22M1-S2Bt 


SFCA 


m 1 6 2 m 


MEMORANDUM FOR 


INSPECTOR CEWeRAL. DEPARTMENT OF DEFENSE, 
400 ARMY NAVY DRIVE. ARLINGTON, VA 22202-4704 


SUBJECT: Response »o the Department ot Defense. Inspector Ger^ral 

(DODIG). Report on DoO Acqutsitioo Woncloroe Count (project no. 
Q2005-DOOOCB-0181), Dated February U. 2006 


This correapooderwe provides the Headquarters, Army ConliBCting * 

{HQ ACA) response to recommerxlation B.1. concerning the subject audit of DoD 
acquisition wortrforce levels. 


The HQ. ACA concurs with the recomroendabon »iat termai Contrae4if>B 
Represerwative (COR) destgnatiorw and contracaor montoly progress reports should be 
requbed. The contract performance periods under these rsootracts •'^d. It^reFors no 
further corrective actions car be taken. Arty future controct requaements tor or 
similar type services, wilt include the proper government and conlrecler reporting 
requirements. 


Sandra O. Sieber 
Director 

Army Contract^ Agency 


KEB-1122 
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Defense Manpower Data Center Comments 


DEPARTMENT OF DEFENSE 
Manpower Daia Center 


jaruri-ODKsa . 

a MO CIlNRaWnnERBY SAY 
vocasuwaROAD 

□ lets K wueH B1.SS. suns 

AlOJKOTSK vnulMA U)M-2UI 

March 13,2006 


MEMORANDUM FOR INSPECTOR GENERAL, DEPARTMENT OF DEFENSE 
SUBJECT; Draft Report on DoD Acquisition Workforce Count (Projea No, D2005-0181) 

Thank you for the opportumty to review and provide comments on your draft 
report, “DoD AoqniHlion Workforce Count,” Pitgect No. D2O05-D0OOCB-O161. DMDC 
concurs in part with the findings and recommendations. 

With regard to Finding A, we believe the report inaccurately reflects DMDC's role 
in providing refined Packard counts. DMDC primarily receives and compiles data from 
the Military Service and OSD agencies and in fact, the Services are aliowKl to modify the 
counts aftw DMDC's inidal antlyds without providing justification for their changes. 

Thus we are not in a position to perform reconciliations of workforce counts nor to “. . . 
explain nor provide written documentation why there were sigmfleant increases and 
decreases for acquiMtion workforce personnel for the Military Servic« and Defense 
agendes ..." WebelteveRecomin^ation AI more prepay recognises the role and 
responsibilities of the Director, Defense Procurement and Aoquisitioa Policy in developing 
and implemeatiag standard procedures vid guidance. 

With regard to Recommendation A.2., that "the Wceew, Defense Manpovw Data 
Cfn f?r develop a knos^edge management prof^am to maintdn coiportf e knowledge 
Defense Manpower Data Center iafomistion systems and processes,” I concur that D^C 
did not fiilly pass along knowledge related to the methods used in attempting to identify 
persormel in the acquisition workforce, and I have taken steps to rectify that situation. 

Should you have any additional questions, ray point of contact for this report is 
Kris Hoffman. Nb. Hoffman can be reaped wa email to lda.hoRman @osd-pentagoii.mil 
or via phone at 831-5B3-2400 (commercial) or 878-2951 ^SN)- 

— ■ 

Robert J. Brandewic 
Director 
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Department of the Air Force Comments 


PEPARTMENT OF TITE AIR FORCE 
WASHINGTON DC 


1 5 MAR Z006 

OI'TICB THE ASSISTANT SliTBETARY 


MEMORANDUM FOR DEPARTMENT OH DKFJ'NSE INSPECTOR GENERAL 

ATTN: DEPUTY INSPECTOR GENERAL FOR AUDI'I ING 

FROM: SAF/AQ 

SUBJECT: Air Force Comments on the Draft Report on the Acavusilion Worteforce (AWF) 
Coynl, DoD IG Code D2005CB-<H81 , 13 Fcbruur>- 2006 

The following is in reply to your metnoniDdum requesting the Assistant Secretary of the 
Air Force (Financial Management and Comptroller) provide Air Force eonunents on subject 
report. 

The DoDlG report listed 2 major Findings. The Air Force concurs with these findings 
without comment. The report also makes 5 recommendatiocs. Tbc Air Force non-conctirs with 
the third reoommondalioa; “We also recammend that the Director revise the instniction to 
estimate contractor equivalents which support the DoD acquisition workforce, and include such 
estimates as Bupplementaiy annual DoD acquisition workforce reporting data to Congress.” 

Hiere is currently no requirement for DoD to report AWF contractor equivalents and 
there is some question to the relevancy of counting contractors. Most contraets are written to 
provide a product or scrvico- the number of ccmtraclors needed to nccompEsh this Mrvrec is not 
relevant to the contract. Fuiftjonnorc, the purpose of the count - support of acquisition 
workforce planning, ccrtificatiMi, and training - is not applicable to support oonuactors. 
Per.sonncl hired under a support contract are assumed to tilready be quaUfied and no further 
certification or training is nece-«ary by the DoD or Services. 

The SAF/AQ point of contact is Major Tony Varkamp. Major Veerkamp can be 
reached at 703-588-7130 or at email: Tcmy.Veertcamp@ponta6“‘.af4nil. 



Military D^uty, Office of the Assistant Secremry 
of the Air Force (Acquisition) 
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Team Members 

The Department of Defense Office of the Deputy Inspector General for Auditing, 
Contract Management prepared this report. Personnel of the Department of 
Defense Office of Inspector General who contributed to the report are listed 
below. 

Richard B. Jolliffe 
Kimberley A. Caprio 
Benjamin A. Mehlman 
Robert M. Sacks 
Linh Truong 
Takia A. Matthews 
Daniel L. Messner 
Cecil B. Tucker 
Loretta L. Loughner 
Meredith Johnson 
Jillisa Milner 


Chairman Levin. We’ll stand adjourned. 

[Questions for the record with answers supplied follow:] 

Questions Submitted by Senator Joseph I. Lieberman 

STATUS OF CURRENT ACQUISITION SYSTEM 

1. Senator Lieberman. Mr. Sullivan and Dr. Gansler, I have several concerns 
with the acquisition system as it currently operates. Most importantly, it seems that 
the requirements validation process under the Joint Capabilities Integration and 
Development System (JCIDS) is not synchronized with the Planning, Programming, 
Budgeting, and Execution (PPBE) funding process. Can any effort at reform succeed 
without addressing this fundamental flaw in acquisition management? 
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Mr. Sullivan. We do not believe that there can be success until there is better 
synchronization between the Department of Defense’s (DOD) requirements, funding, 
and acquisition processes. The Government Accountability Office (GAO) has pre- 
viously reported that the lack of integration between the JCIDS and PPBE proc- 
esses is a key factor that contributes to the Department’s inability to achieve a bal- 
anced portfolio of weapon system programs that meets the needs of the joint 
warfighter and matches needs with available resources. The JCIDS process has not 
been effective in prioritizing needs from a joint. Department-wide perspective, and 
it largely approves capability needs without accounting for the resources or tech- 
nologies that will be needed to acquire the desired capabilities. Resource allocation 
decisions in DOD take place through the PPBE process, which is separate from 
JCIDS. PPBE largely allocates resources on a Service-by-Service basis and does not 
effectively link resources to capabilities. In addition, the process allows too many 
programs to start development with unreliable cost estimates and without a com- 
mitment to fully fund them. Until DOD establishes a more integrated approach to 
weapon system acquisition, it will continue to struggle to effectively prioritize 
warfighting needs, make informed trade-offs, and achieve a balanced mix of weapon 
systems that are affordable, feasible, and provide the best value to the warfighter. 

Dr. Gansler. I believe “cost” should be an essential part of the “requirements 
process” — since, in a resource-constrained environment, it directly determines “num- 
bers” (of systems procured) and that determines force effectiveness (even more than 
individual weapon’s effectiveness). 

2. Senator Lieberman. Mr. Sullivan and Dr. Gansler, what are your recommenda- 
tions for effectively balancing the requirement validation process with the calendar- 
driven funding process? 

Mr. Sullivan. To improve DOD’s ability to deliver weapon systems at the right 
time and right cost, GAO has recommended that the Department implement an en- 
terprise-wide portfolio management approach to making weapon system investments 
that integrates the determination of warfighting needs with available resources and 
cuts across the Services by functional or capability areas. Such an approach would 
focus on assessing weapon system investments collectively from an enterprise level, 
rather than as independent and unrelated initiatives. By following a disciplined, in- 
tegrated process — during which the relative pros and cons of competing weapon sys- 
tem proposals are assessed based on strategic objectives, warfighter needs, and 
available resources, and where tough decisions about which investments to pursue 
and not to pursue are made — DOD could minimize duplication between Service com- 
ponents and more effectively support each new development program it commits to. 
Furthermore, to ensure effective weapon investment decisions are made, GAO has 
recommended that a single point of accountability be established at the Department 
level with the authority, responsibility, and tools to implement portfolio manage- 
ment. 

Dr. Gansler. My answer to question #1 (above) will greatly aid in this; but, in 
general, the “requirements process” must look well beyond the near-term budget 
(funding) process. Nonetheless, the frequent budget changes (particularly, by Con- 
gress) introduce great instability and inefficiency into the implementation of the 
weapons acquisition process. 


JOINT requirements 

3. Senator Lieberman. Mr. Sullivan and Dr. Kaminski, I am concerned that DOD 
has historically done a poor job of procuring capabilities that address joint require- 
ments, and more specifically, that the Services have little incentive to spend what 
they see as their limited budgets on capabilities that will principally benefit sister 
Services. One particular instance of this is the Multi-Platform Radar Technology In- 
sertion Program, and the Air Force’s apparent reluctance to procure a system that 
will principally serve the Army’s battlefield management requirements. How can the 
acquisition process better prioritize and procure such crucial joint capabilities? 

Mr. Sullivan. GAO has recommended that DOD should determine and allocate 
appropriate resources for more effective joint capabilities development planning. The 
functional capabilities boards, which were established to manage the JCIDS process 
and facilitate the prioritization of needs, have not been staffed or resourced to effec- 
tively carry out these duties. Similarly, the combatant commands (COCOMs) also 
lack the analytic capacity and resources to become more fully engaged in JCIDS. 
GAO recently reviewed JCIDS documentation related to new warfighting capabili- 
ties and found that most — almost 70 percent — were sponsored by the military Serv- 
ices with little involvement from the joint community, including the COCOMs, 



177 


which are responsible for carrying out military missions. The Services drive the de- 
termination of capability needs, in part because they retain most of DOD’s analyt- 
ical capacity and resources for requirements development. 

Within the Department’s PPBE process, the individual military Services are re- 
sponsible for budgeting and allocating resources under authority that is commonly 
understood to be based on title 10 of the United States Code — to organize, train, 
and equip military forces. In this structure, the budget is based more on individual, 
service-focused needs than on joint warfighting needs. In the past, GAO has re- 
ported that this situation has contributed to interoperability problems among weap- 
on systems and unnecessary duplication of capabilities in some areas. The Office of 
the Secretary of Defense (OSD) reviews and makes adjustments to the Services’ 
budgets, but this takes place toward the end of the PPBE cycle when it can be dif- 
ficult and disruptive to make changes, such as moving funds to higher-priority pro- 
grams or to support joint needs. DOD has recently taken steps to establish capa- 
bility portfolio management in selected areas in an attempt to overcome the service- 
centric nature of the resource allocation process. However, because the portfolios are 
largely advisory and lack authority and control over the allocation of resources, 
there may be limits to achieving joint capabilities. 

Dr. Kaminski. The priority must be set in the budget process, with key oversight 
from OSD and the Joint Chief of Staff. The procurement can be improved by first 
developing (a several year process) then assigning key program management per- 
sonnel with the requisite training and domain experience. Alignment of the respon- 
sibility, authority, and accountability of the program manager will require that a 
degree of trust be established between the program manager and those responsible 
for our oversight mechanisms. We must be prepared to delegate authority to the 
program manager, and provide him or her with some flexibility to manage — to ad- 
just levels and allocation of funding, to adjust the allocation of performance param- 
eters, to adjust schedule, and to tailor the acquisition approach to be responsive to 
the need. Clearly, there must be bounds established beyond which the program 
manager must seek approval from oversight authorities. But I believe these bounds 
are too narrow and inflexible today. We also need to provide sufficient upfront fund- 
ing, and maintain funding stability throughout program execution. 

4. Senator Liebeeman. Mr. Sullivan and Dr. Kaminski, are either the JCIDS or 
PPBE as currently constructed able to overcome parochial service interests in this 
regard? 

Mr. Sullivan. No, I do not believe that the JCIDS and PPBE processes are con- 
structed effectively to address the service-centric manner in which weapon systems 
are acquired in DOD. Past studies chartered by DOD and other organizations have 
reported similar conclusions and made a number of recommendations to improve the 
acquisition of joint warfighting capabilities. 

Dr. Kaminski. Both help, but I don’t believe they are sufficient. 


DEPARTMENT OF DEFENSE SCIENCE AND TECHNOLOGY PROGRAM 

5. Senator Lieberman. Dr. Gansler and Dr. Kaminski, as we try to reduce risk 
in acquisition programs, what role does (or should) DOD science and technology 
(S&T) program play? 

Dr. Gansler. S&T are critical to both “staying ahead” (which is the basis of our 
national security strategy of “technological superiority”) as well as to risk reduc- 
tion — since we need to prove out the S&T first, before introducing it into weapons 
acquisition. 

Dr. Kaminski. With good S&T programs supporting development planning, we can 
assign managers to develop prototypes, critical systems or components needed to 
better understand cost/performance trades and reduce risk. With the appropriate 
technology base in place, it is reasonable to expect that many of these developments 
can be completed in 2-4 years, so one manager will be in place from start to deliv- 
ery of those critical subsystems during the development planning phase. This should 
allow the full system development to proceed on a shorter schedule as a result of 
the risk reduction. 

6. Senator Lieberman. Dr. Gansler and Dr. Kaminski, do you think that the De- 
partment’s S&T programs, and in particular the DOD laboratories, are sufficiently 
resourced and staffed to provide support to the acquisition programs and address 
the problems you are discussing today? 

Dr. Gansler. The DOD’s S&T program must be adequately funded to “stay 
ahead”; and, traditionally, whenever there is a shortage of DOD money it is one of 
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the areas the Services tend to cut (since it is long-term). Thus, in the likelihood of 
a coming budget crunch, it must be protected — not just in the DOD labs, but par- 
ticularly in the universities and in industry. 

Dr. Kaminski. No. We are short in both quantity and quality of staffing, and are 
not using the latest tools to support early modeling, simulation, and testing. 

7. Senator Lieberman. Dr. Gansler and Dr. Kaminski, what recommendations do 
you have for this committee to improve the quality of the DOD S&T program and 
the quality of the DOD laboratories? 

Dr. Gansler. Adequate funding, and allowance for foreign graduate students, 
scholars, and researchers (per Presidential Decision Directive- 189) to work on DOD 
S&T, along with “peer review” of all proposals, are critical to the United States stay- 
ing ahead. The SBIR and STTR programs are also a great benefit, and must be 
maintained. 

Dr. Kaminski. The first step is to ensure that we not only restore, but also en- 
hance early and continuing systems engineering coupled with the effective develop- 
ment planning needed to drive our S&T program. This will require commitment of 
more significant investment dollars earlier in our acquisition programs, and a com- 
mitment to build a cadre of systems engineers and development planners with the 
education, training, and domain experience needed to be effective. Attracting “best 
and brightest” to this work — and keeping them — will require a personnel system 
that will identify and track these important human resources and establish a career 
path to allow those who are successful to advance to senior program management 
and leadership positions. Their domain experience will be enhanced by managing 
the building of critical technology demonstrators and subsystems during the devel- 
opment planning program, reducing risk, and building skills and experience at the 
same time. 


Question Submitted by Senator Jack Reed 

S. 454 legislation 

8. Senator Reed. Dr. Gansler, in the proposed S. 454 legislation, section 203 fo- 
cuses on meiximizing competition throughout the life cycle of major acquisition pro- 
grams. One proposed provision is that modular, open architectures are used to en- 
able competition for upgrades. Please comment on how the implementation of this 
recommendation would positively impact major weapons programs that have been 
sole sourced for decades, and how the Services can rapidly implement these changes 
to achieve maximum benefits. In addition, please comment on the feasibility of ap- 
plying this provision to current programs, not just new programs. 

Dr. Gansler. Let me begin by noting that the objective should not be “maximizing 
competition”; rather, it should be “maximizing the effectiveness of competition.” 
Competition for its own sake, can be detrimental. For example, there must always 
be the “option” of competition, but if a firm is constantly improving performance and 
lowering costs, they should be rewarded by not recompeting the program; but there 
must always be the option of introducing competition if costs are rising (so the 
“threat” of competition is what creates the incentive for lowering costs; and the fol- 
low-on is the reward). 

Open architectures are clearly one way to allow competition to be introduced at 
the subsystem level (which often is 70-80 percent of the total weapons cost); so it 
should be greatly encouraged — both on upgrades of current systems and on new 
ones. 


Questions Submitted by Senator Daniel K. Akaka 
weapon systems acquisition reform act 

9. Senator Akaka. Mr. Sullivan, section 105 of the Weapon Systems Acquisition 
Reform Act recommends the Joint Requirement Oversight Council (JROC) seeks 
input from the combatant commanders in identifying joint military requirements. 
Currently, the combatant commanders produce an annual Integrated Priority List 
(IPL) that attempts to satisfy this same goal. In your prepared testimony, you stat- 
ed some of the combatant commanders do not feel their needs are sufficiently ad- 
dressed. What must be done within DOD to fully implement the intent of this legis- 
lation to ensure combatant commander’s influence throughout the acquisition sys- 
tem to decrease the Service-centric environment we’ve seen for years? 
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Mr. Sullivan. GAO and several other studies have recommended that DOD in- 
crease joint analytic resources for a less stovepiped understanding of warfighting 
needs. While each COCOM submits an IPL to the Chairman of the Joint Chiefs of 
Staff each year, which defines the COCOM’s highest-priority needs, it does not ap- 
pear that the COCOM needs are well integrated with JCIDS. According to officials 
from several COCOMs that we have met with, needs identified through IPLs are 
not typically developed into JCIDS capability proposals. Some COCOM officials 
pointed out to us that because of their limited resources, they must pick and choose 
capability needs and persuade one or more of the military Services to sponsor a pro- 
posal that addresses the COCOM’s need. The Department has been working to give 
more time and priority to the commanders’ IPLs, but more attention is needed. 

10. Senator Akaka. Mr. Sullivan, in your prepared testimony, you mentioned the 
establishment of the DOD capability portfolio management process. DOD identified 
nine joint capability area portfolios to mitigate the tendency to develop Service-cen- 
tric, stovepiped solutions. However, the capability portfolio managers lack the nec- 
essary authority and control required to be successful. How can DOD reform their 
capability portfolio management process to mitigate the tendency to develop Service- 
centric, stovepiped solutions? 

Mr. Sullivan. DOD’s capability portfolios are intended to advise the Deputy Sec- 
retary of Defense on how to optimize investments within their respective capability 
areas. They have no independent decisionmaking authority and must operate within 
the Department’s existing framework for acquiring weapon systems. That is, capa- 
bility needs are still determined through JCIDS and resources are allocated through 
PPBE. Although it is too soon to assess the impact of capability portfolio manage- 
ment, according to some DOD officials, the portfolios provided key input and rec- 
ommendations in the 2010 budget process. However, without portfolios in which 
managers have authority and control over resources, DOD is at risk of continuing 
to acquire weapon systems in a stovepiped manner. Ultimately, DOD needs to de- 
velop an integrated portfolio approach that brings the determination of require- 
ments together with the allocation of resources. To be effective, such an approach 
must have committed leadership, empowered portfolio managers, and accountability 
at all levels of the Department. 

11. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, changes made 
to a system after acceptance of review cost many times more than the same changes 
made during the requirements gathering phase. How can we improve the require- 
ments gathering phases of the acquisition of major weapons systems to front-load 
these changes to achieve significant cost savings? 

Dr. Gansler. Please refer to my answer to question #1. If cost is a requirement, 
then changes that come in must not only have showed how they will improve per- 
formance but how they will live within the cost requirement. 

Clearly, the earlier this is done, the better; but technology changes, threat 
changes, et cetera, do come along, and as long as they are explicitly within the over- 
all cost requirement they can be allowed in. 

Dr. Kaminski. Requirements adjustments during execution of acquisition pro- 
grams are not necessarily bad if they are conducted within the framework of value 
propositions. Good program management requires continuing adjustment of the re- 
quirements flow-down and allocation to major subsystems based upon expected per- 
formance, schedule, and cost. Even top level requirements require continuing assess- 
ment and adjustment based upon similar value propositions. As we discover more 
about costs and performance during execution, good program managers should be 
interacting with requirements developers to adjust as needed to provide the best 
value for the user. This requires some flexibility in cost allocation, and it also places 
demands upon requirements developers to become involved in tradeoff decisions. To 
be effective, requirements developers as well as acquisition managers must have ex- 
perience with systems engineering tools and techniques, and both must have suffi- 
cient domain experience to engage productively in cost-performance tradeoffs. 

Mr. Adolph. A disciplined analysis of alternatives process should be employed to 
support the JCIDS and acquisition processes, from capability needs identification to 
include system design and development as well as life-cycle improvement. Emphasis 
must be placed on improving the processes for relating mission effectiveness and 
cost to system design, system performance, and suitability. Effective feedback proc- 
esses are of special importance for spiral developments to identify enhancements 
which will improve performance and suitability. Improving the quality and speed of 
this feedback is increasingly important in responding to rapid changes in threat en- 
vironments. 
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12. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, significant cost 
overruns are incurred from the instability of mission and design requirements dur- 
ing testing and even, in some cases, production. To what extent do you think it 
would be advantageous to more rigorously enforce a freeze on fundamental require- 
ments at the system requirements review gate? 

Dr. Gansler. The greatest causes of cost growth, after program initiation, are 
budget changes (often caused by Congress) and “requirements” changes — both of 
which can be controlled by discipline. (See question #11.) 

Dr. Kaminski. It is always beneficial to have a firm understanding of very top 
level requirements at the start of a program. But adjustments of subordinate re- 
quirements during execution of acquisition programs are not necessarily bad if they 
are conducted within the framework of value propositions. Even top level require- 
ments need continuing assessment based upon evolving value propositions. As we 
discover more about costs and performance during execution, good program man- 
agers should be interacting with requirements developers to adjust as needed to pro- 
vide best value for the user. This requires some flexibility in cost allocation, and 
it also places demands upon requirements developers to become involved in tradeoff 
decisions. To be effective, requirements developers as well as acquisition managers 
must have experience with systems engineering tools and techniques, and both must 
have sufficient domain experience to engage productively in cost-performance trade- 
offs. 

Mr. Adolph. Rigorous enforcement of key requirements thresholds should be the 
norm when entering system development and demonstration. There needs to be 
more rigor and discipline in the entire requirements definition and acquisition proc- 
ess. Issues that need to be addressed in relation to requirements setting include 
technology readiness (see response to question 13 below), the translation of require- 
ments into design criteria, with attention to testability at the subsystem and system 
levels, as well as defining thresholds for key performance parameters. 

13. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, cost overruns are 
also caused by the inherent unpredictability of the development of new technologies 
in the course of fulfilling system contracts. Would you recommend restricting the 
proposed designs for weapons systems to proven technologies while adequately fund- 
ing contractual vehicles for pure research? 

Dr. Gansler. I would restrict the first “block” of the system to proven technology; 
then — using “spiral development” — evolve the system’s future blocks as future tech- 
nology is proven out; and as long as the future technology either improves perform- 
ance (at the same cost) or lowers the cost. This future technology should be funded 
in parallel with the system, until it is proven. 

Dr. Kaminski. This will he helped by better development planning and alignment 
of incentives. With good development planning, we can assign managers to develop 
prototypes, critical systems or components needed to better understand cost/per- 
formance trades and reduce risk. It is reasonable to expect that many of these devel- 
opments can be completed in 2-4 years, so we do not need to restrict ourselves to 
proven technology at the beginning of development planning. As these critical sub- 
systems are delivered and tested, the risk reduction and domain experience gained 
in both government and industry will allow us to prove the critical technologies in- 
volved and reduce the time required to develop, integrate, and test the full system. 
We can rely on the experience gained during development planning to apply in- 
formed judgment to adjust requirements to improve value, reduce time, and better 
estimate and manage costs. 

Mr. Adolph. Weapon system design should be based on proven technologies. His- 
tory has shown repeatedly that doing otherwise introduces unacceptable risk in to 
weapons system development which necessarily translates into cost growth and 
schedule delays. Assessing whether a technology is sufficiently mature is currently 
accomplished through the Technology Readiness Levels (TRL) construct which 
should employ government test and evaluation (T&E) as the key component in mov- 
ing a technology from TRL 4 to TRL 6. Recent consideration has been given to 
strengthening early involvement of the test community by reinvigorating the gov- 
ernment developmental test process and by involvement of OT&E in readiness as- 
sessments. These are sound initiatives and should be focused on providing rigor to 
the TRL assessments in terms of the test methodology as well as the test environ- 
ment. Implementation of this more disciplined approach will require a reversal to 
the trend of cutting back on government test personnel as well as strengthening 
their role in supporting the acquisition process. In that regard, the Director of De- 
velopmental Test and Evaluation should make an assessment of technological matu- 
rity and integration risk, based on test results and report the findings to the Under- 
secretary of Defense for Acquisition, Technology and Logistics. 
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There may be rare exceptions when a requirement is so pressing that risks must 
be taken to design a system which incorporates an unproven technology. In those 
instances, a high-priority, adequately funded, and closely monitored effort must be 
undertaken to mature the technology. This must include an early, rigorous assess- 
ment of the difficulties associated with the development effort as well as testing to 
fully assess the requisite maturity. 

Additionally, the TRL process has been focused on the technology of system com- 
ponents. Problems with manufacturing these components in production quantities 
have also led to significant cost growth for weapons systems. Previous legislation 
established the Manufacturing Technology Program to identify and develop initia- 
tives to improve manufacturing quality, productivity and technology. Consideration 
should be given to expanding the TRL process to evaluate the maturity of the pro- 
duction methods by which parts are manufactured. 


Questions Submitted by Senator Bill Nelson 

MODELING AND SIMULATION 

14. Senator Bill Nelson. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, computer 
modeling and simulation (M&S) tools can be very useful in discovering problems in 
major systems before we spend a lot of money or time on hardware development. 
DOD has some programs in M&S — which in my opinion are underfunded and dis- 
jointed. What is your assessment of M&S activities within DOD and their ability 
to address some of the acquisition problems we are discussing here today? 

Dr. Gansler. M&S can be a great aid in system design and performance evalua- 
tion, and should be an important element in the development of all systems. 

Dr. Kaminski. M&S can have a very significant positive impact on acquisition per- 
formance if there is sufficient emphasis on continuing validation of M&S tools and 
integration with T&E. Giving early and serious attention T&E will require strength- 
ening our T&E organizations and personnel. T&E is often an afterthought, and con- 
tracts are often written without any mention of how we will test the product. We 
spend large amounts of money when a large development team waits for test re- 
sults. The alternative is to spend less money and time by early development of M&S 
tools, continuously validating the models with testing at critical points, and consid- 
ering the investment in M&S and test resources as a critical part of our systems 
engineering and development planning efforts. 

Mr. Adolph. With some exceptions, the capabilities and limitations of simula- 
tions, simulators and physics-based models of individual subsystems are well under- 
stood. These tools are generally used where appropriate. However the use of mod- 
eling to support requirements definition, analysis of alternatives, and test design 
needs to be pursued more vigorously. The expectations regarding the use of M&S 
to support elements of the acquisition process, including system integration, T&E, 
and manufacturing have increased along with advances in computer technology. 
Distributed T&E (and training) using multiple ranges, laboratories, and facilities 
have benefited from the use of live, virtual, and constructive simulations. However, 
the availability of adequate models and modeling tools involving human dynamics 
for use in T&E, mission planning, systems engineering, and risk assessment has not 
kept pace. 

M&S to support acquisition/test programs has been the norm for decades, and re- 
ceived additional impetus in the 1990s with advances in information processing 
technology. In many cases, the incremental improvements as related to acquisition 
were oversold in the late 1980s and 1990s. Additionally, the requirements for 
verification, validation, and accreditation have proven to be more costly and 
timeconsuming than anticipated. Many of the M&S activities initiated in the last 
15 years were disjointed, with few successful attempts to coordinate requirements, 
build and share common models, correlate performance, and ensure that the models 
were adequate for the intended applications. Numerous recent M&S developments 
have been terminated after spending millions because of the failure to produce a 
useful product. 

Building models to meet every need is not feasible. An overall M&S vision is re- 
quired to identify where efforts should be focused, and to ensure that new efforts 
are within that vision. If high priority M&S efforts within that vision could be iden- 
tified, it would be possible to establish requirements, understand what is currently 
available, and then determine shortfalls in M&S capability. One approach is exem- 
plified by the Testing in a Joint Environment Roadmap, which was recently devel- 
oped by DOT&E and approved by the Deputy Secretary of Defense. While focused 
on T&E usage, this capability would meet a significant percentage of contractor de- 
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velopment and testing M&S requirements. The T&E roadmap identifies a require- 
ment for a distributed live, virtual, constructive T&E capability; largely supported 
by existing models of friendly and threat systems. Much of this capability could 
come from existing models of acquisition systems developed by program contractors, 
in conjunction with threat models available from the intelligence agencies. However, 
there has been virtually no effort to identify requirements for high priority joint 
missions, to determine what friendly and threat force representations are required, 
assess the availability of existing models that meet those requirements, and then 
use that information to define M&S shortfalls. Additionally, there are insufficient 
mechanisms and processes to feed back data from operational tests and field exer- 
cises to further validate and refine models. Archiving mechanisms that can store 
and locate data for future applications are needed. 

There are also supporting M&S requirements (engineering and physics-based 
models) which are required in the validation of higher-level engagement models. 
Higher level models, such as envisioned in the T&E roadmap, could be used to iden- 
tify areas with the greatest risk and uncertainty to better define areas where engi- 
neering and physics-based models are required. The initial effort should be to review 
the past studies on M&S in the DOD and determine those requirements that could 
contribute the most to improving M&S capability. While most studies have identical 
or similar recommendations, the majority of those recommendations have not been 
implemented. 

15. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, how would you 
recommend addressing shortfalls in DOD activities in M&S? 

Dr. Gansler. To address shortfalls, I would ask an independent group (such as 
the Defense Science Board) for an assessment and a set of recommendations (includ- 
ing appropriate funding). 

Dr. Kaminski. We need upfront investment in M&S tools to understand the value 
propositions and the tradeoffs among cost, schedule, and performance of the mis- 
sion. We spend large amounts of money when a large development team waits for 
test results. The alternative is to spend less money and time by early development 
of M&S tools, continuously validating the models with testing at critical points, and 
considering the investment in M&S and test resources as a critical part of our sys- 
tems engineering and development planning efforts. 

Mr. Adolph. Focused M&S business plans must be developed. Separate plans are 
needed for requirements definition, acquisition/test, training and force structure 
evaluations. As discussed above, the latter plans should be used to identify critical 
joint missions and provide a basis for prioritizing M&S requirements; to understand 
current capabilities and identify major shortfalls. 

An issue that needs to be addressed is the availability of existing models to meet 
current requirements. Many models were developed by system contractors to meet 
specific needs. In some cases they are proprietary and there are rarely provisions 
for models to be maintained current so they represent a system in the field with 
normal improvements and modifications, or changes to the treats. Often contractors 
are not funded to make their models generally available. The government must ad- 
dress issues relating to long-term configuration control, upgrades, ownership, and 
funding throughout the system lifecycle, to include training. In addition, addressing 
the requirements for a single joint mission doesn’t capture requirements across sev- 
eral potential joint missions and scenarios. 

However, it would provide a starting point for future requirements and identify 
the most sensitive issues in implementation of the overall concept. Addressing one 
major joint mission scenario would identify a large percentage of requirements for 
other joint missions. 

Finally, an important consideration is the physical location of major simulation 
laboratories and related facilities. There are often valid reasons to locate a simula- 
tion capability at a contractor development facility to support initial development; 
however it places limitations on the future utility and accessibility of the simulation, 
as well as the need to replicate parts of the simulation capability at a government 
test facility or logistics center. Prior to making a decision regarding the location of 
simulation facilities for each major program, an assessment should be made as to 
the most cost-effective locations(s), taking into account follow-on and future pro- 
grams. This decisionmaking process should include the appropriate Service MRTFB 
representatives, as well as the TRMC. (See also my response to question #17.) 
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TEST AND EVALUATION ENTERPRISE 

16. Senator Akaka. Mr. Adolph, a major step in developing weapons systems is 
ensuring that they are appropriately tested during their development and before 
their operational use. I believe that DOD is letting its testing enterprise decline pre- 
cipitously in terms of testing equipment, infrastructure, and workforce. What role 
do you feel that the DOD testing enterprise plays in ensuring acquisition programs 
deliver their systems on schedule and on budget? 

Mr. Adolph. The role of the DOD testing enterprise is addressed in detail in the 
recent Defense Science Board Task Force Report on Developmental T&E. My writ- 
ten testimony covers the major findings and recommendations from that study, and 
while it emphasizes the impacts of reliability, availability, and maintainability on 
life-cycle costs, the implementation of some acquisition reform initiatives diminished 
the role of government T&E in the acquisition process. 

A brief summary response follows. The traditional role of the government during 
the test planning phase included the identification of test resource requirements and 
test facilities, the development of a test strategy and detailed T&E plans. When pro- 
grams moved from the planning phase to the execution phase, the government tradi- 
tionally performed operational assessments, participated in test conduct and anal- 
ysis, and performed an evaluation of the test results for the government Program 
Office. With a few exceptions, this is no longer the case. In many instances the gov- 
ernment testing enterprise has gone from vigorous test participation to oversight to 
out-of-sight. 

As a minimum, government test organizations should reconstitute and retain a 
cadre of experienced T&E personnel to perform the following functions: 

• Participate in the translation of operational requirements into contract 
specifications and key performance parameters 

• Participate in RFP preparation and source selection, including assurance 
that contractor developed models and simulations are available to the gov- 
ernment 

• Participate in technology maturity determination and prototype evalua- 
tions 

• Perform operational assessments in conjunction with the systems engi- 
neering process 

• Participate in developmental T&E planning and technical reviews 

• Participate in test conduct, analysis, reporting capabilities, and limita- 
tions, with emphasis on government analysis and reporting 

17. Senator Akaka. Mr. Adolph, what is your assessment of the state of the DOD’s 
testing facilities and their ability to perform their designated missions? 

Mr. Adolph. The focus of DOD’s testing facilities is the Major Range and Test 
Facility Base (MRTFB) which is comprised of those ranges and facilities designated 
most critical to supporting the T&E needs of DOD development and acquisition pro- 
grams. Since the 1970s, DOD’s intent has been to ensure the capabilities and capac- 
ities of these MRTFB facilities/ranges meet the T&E requirements of emerging tech- 
nologies and weapon system programs, taking into consideration the long-lead times 
necessary to acquire these capabilities, which often dictates that new test tech- 
nologies be developed and matured. Likewise, a skilled T&E workforce must be in 
place to address the challenges of testing and evaluating new weapon systems with 
increasing complexities of technologies, integration, and missions. Given that this is 
the intention of the MRTFB, where do we stand today? 

As I noted in my written statement, several changes resulted from the implemen- 
tation of acquisition reform initiatives in the late 1990s. Among them was a de-em- 
phasis on the use of government test facilities, testers, and evaluators. When pro- 
grams need to use government test facilities, the first inclination has sometimes 
been to use non-MRTFB capabilities to avoid reimbursement costs required by the 
MRTFB charge policies, often using training ranges having little or no T&E infra- 
structure. If the MRTFB facilities/ranges are used, programs typically prefer to re- 
quest only the raw data to perform their own evaluations. Consequently, the Serv- 
ices have been reducing their budgets and T&E workforce for sustaining the 
MRTFB. Not only has the MRTFB experienced loss of a large number of the experi- 
enced subject matter engineering experts, scientific, and mathematical personnel, 
several MRTFB test facilities are being mothballed or closed. The government skills 
to restore or replace them will be lost over time. 

Investments in new test capabilities in the MRTFB for anticipated weapon system 
technologies and test programs have been increasingly awaiting the first program 
to need them for funding rather than using T&E institutional funds. This practice 
is counter to the intent of having an MRTFB; i.e., anticipating and having 'T&E ca- 
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pabilities in place when new development or acquisition programs require them. 
Some new T&E capabilities require long-lead times to plan, budget, and to develop 
to be ready when needed by a program, thus can cause the test program schedules 
to slip, inadequate capabilities to be accepted as work-arounds, or not to be tested 
at all. Furthermore, these facilities are normally specialized for the programs paying 
for them, and may have little or no residual use for other programs, particularly 
when they are co-located with the prime contractor’s manufacturing facility. While 
the Test Resources Management Center (TRMC) “Strate^c Plan for DOD Test and 
Evaluation Resources” serves as a good guide to the actions required to sustain a 
capable MRTFB, the TRMC has virtually no power to require the military depart- 
ments to sustain adequate operating or investment funding beyond the funds man- 
aged through the Central Test and Evaluation Investment Program and T&E/S&T 
Program. As a result, my assessment is that in some instances, the MRTFB no 
longer meet its intended purpose of having adequate capabilities and capacity avail- 
able for future acquisition programs. 


Questions Submitted by Senator E. Benjamin Nelson 

FISCAL responsibility 

18. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Ad- 
olph, cost overruns and fiscal irresponsibility are great concerns for me. During his 
hearing in January, the Secretary of Defense stated that many programs that cost 
more than anticipated are built on an inadequate initial foundation, which in my 
opinion begins with defining requirements. It has been reported that many of the 
COCOMs do not believe their requirements are sufficiently represented by the cur- 
rent process, which seems to be centered on each Service, not the Armed Forces as 
a whole. This results in duplication of capabilities, non-alignment of requirements 
to capability, and interoperability problems, all of which lead to expensive require- 
ments creep. Should COCOMs play a lead role in defining needed capabilities? 

Mr. Sullivan. Yes, we believe that the COCOMs should play a stronger role in 
defining capability needs. Requirements determination in DOD, particularly for 
major weapon system programs, continues to be driven largely by the military Serv- 
ices. Our work has revealed that although the COCOMs — which are responsible for 
planning and executing military missions — are the principal joint warfighting cus- 
tomer in DOD, they have played a limited role in determining requirements. In ana- 
lyzing requirements documentation related to new capability proposals, we recently 
reported that most — almost 70 percent — were sponsored by the military Services 
with little involvement from the COCOMs. Other studies have also raised concerns 
that the Services and the COCOMS do not routinely collaborate to identify possible 
joint capability solutions. 

• The Defense Acquisition Performance Assessment Panel concluded that 
JCIDS resulted in capabilities that did not meet warfighter needs in a time- 
ly manner and recommended that JCIDS be replaced with a COCOM-led 
requirements process in which the Services and defense agencies compete 
to provide solutions. 

• The Defense Science Board similarly reported that JCIDS has not pro- 
vided for increased warfighter influence, but instead actually suppresses 
joint needs in favor of military Service interests, and recommended an in- 
crease in the formal participation role of the COCOMs in the JCIDS proc- 
ess. 

• The Center for Strategic and International Studies has also recommended 
that the Joint Forces Command take the lead in conducting capabilities de- 
velopment planning for the COCOMs and become a formal member of the 
JROC. 

By continuing to rely on capability needs defined primarily by the Services, DOD 
may be losing opportunities for improving joint warfighting capabilities and reduc- 
ing the duplication of capabilities. 

Dr. Gansler. The Packard Commission and (subsequently) Goldwater-Nichols 
stated that all requirements were “joint” (since that is the way we fight). The JROC 
was set up to represent the COCOMs in the “requirements process.” Interoperability 
should always be a critical design and test requirement. The COCOMs should be 
the source of requirements for both “urgent needs” and for JCTDs. 

Dr. Kaminski. I believe that the COCOMs should play a strong role. But to do 
so, they need to develop the expertise needed to engage in cost, schedule, and per- 
formance tradeoffs. Even top level requirements need continuing assessment based 
upon evolving value propositions. As we discover more about costs and performance 



185 


during execution, good program managers should be interacting with requirements 
developers to adjust as need to provide best value for the user. This requires some 
flexibility in cost allocation, and it also places demands upon requirements devel- 
opers to become involved in tradeoff decisions. To be effective, requirements devel- 
opers as well as acquisition managers must have experience with systems engineer- 
ing tools and techniques, and both must have sufficient domain experience to en- 
gage productively in cost-performance tradeoffs. 

Mr. Adolph. I don’t have the background or experience to respond to this ques- 
tion. 

19. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Ad- 
olph, how successful are DOD’s processes for matching warfighter needs with re- 
sources? 

Mr. Sullivan. GAO has previously reported that DOD’s inability to achieve a bal- 
anced portfolio of weapon system programs that matches joint warfighting needs 
with available resources is due in part to the fragmented structure of the Depart- 
ment’s requirements, funding, and acquisition processes. The JCIDS process has not 
been effective in prioritizing needs from a joint. Department-wide perspective, and 
it largely approves capability needs without accounting for the resources or tech- 
nologies that will be needed to acquire the desired capabilities. Resource allocation 
decisions in DOD take place through the PPBE process, which is separate from 
JCIDS. PPBE largely allocates resources on a Service-by-Service basis and does not 
effectively link resources to capabilities. In addition, the process allows too many 
programs to start development with unreliable cost estimates and without a com- 
mitment to fully fund them. Until DOD establishes a more integrated approach to 
weapon system acquisition, it will continue to struggle to effectively prioritize 
warfighting needs, make informed trade-offs, and achieve a balanced mix of weapon 
systems that are affordable, feasible, and provide the best value to the warfighter. 

While DOD recently reported that the JROC is doing more to seek out and con- 
sider input from the COCOMs through regular trips and meetings to discuss capa- 
bility needs and resourcing issues, we found that many of the COCOMs still do not 
believe their needs — reflected through the IPL process — are sufficiently addressed 
through DOD’s requirements process. In order to grant the COCOMs more influence 
in identifying requirements, DOD should consider providing the COCOMs with ad- 
ditional resources to establish robust anal 3 d;ical capabilities for identifying and as- 
sessing their capability needs. In addition, DOD and the Joint Staff should ensure 
the JROC gives equal consideration to COCOM needs. 

Dr. Gansler. As Secretary Gates has stressed, we need a better “balance” be- 
tween likely warfighter needs and resource allocations. Today, the budgets are un- 
balanced in the direction of potential, future peer competitors. 

Dr. Kaminski. The success is mixed. Warfighters are often very able to address 
their immediate needs, but addressing future needs can be challenging. Warfighters 
also need to be open to new conops — new approaches to performing their mission 
that can be derived from new, enabling technologies. Similarly, developers and cost 
estimators are more able to address near-term estimates involving technologies that 
are mature and familiar, and less able to do so with future technologies that are 
less mature and less familiar. Matching needs and resources must be a continuing, 
iterative process. This process must have a solid foundation in systems engineering, 
and the systems engineering must in turn have some foundation in M&S which is 
validated by building and testing. Effective value proposition assessments will occur 
only with skilled, trained, and experienced personnel from requirements and acqui- 
sition communities working together within a systems engineering framework to re- 
fine needs and solutions. 

Mr. Adolph. I don’t have the background or experience to respond to this ques- 
tion. 


TECHNOLOGICAL CHANGE AND WEAPONS SYSTEMS 

20. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Ad- 
olph, one of the challenges we face with new technology and the rapid advancement 
of technology is the desire to outfit our military men and women with the latest and 
most advanced equipment. While we all want to provide our military with the latest 
advances in technology, I worry that as technology changes we are in an ever state 
of trying to upgrade a system while it is in development. How can we best handle 
constant technology changes for weapons systems that take years to develop and de- 
liver? 
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Mr. Sullivan. First, technology development must be placed on the critical path 
of any new acquisition program. Technology development requires trial and error, 
discovery, and invention, which should all be done in a S&T environment before 
technologies are brought onto programs. Second, GAO has found that successful 
commercial companies use an evolutionary or incremental approach that reduces de- 
velopment risks and enables quicker delivery of products to the customer. These 
commercial companies have implemented an evolutionary approach by establishing 
time-phased plans to develop a new product in increments (5 years or less) based 
on technologies and resources achievable now and later. They set requirements for 
products that require only technologies that they know will work, and wait to put 
more risky technologies on future increments once their risks have been diminished. 
An open architecture approach to development is also important so that product 
components, such as technologies, can be upgraded or added in subsequent product 
development increments, without having to do a major redesign of the product. 

Historically, DOD’s approach to developing new weapon systems often attempts 
to satisfy the full capability in a single step, “big-bang” approach regardless of the 
design complexity or the maturity of the technologies. Under this approach, the 
warfighter can wait over a decade to receive any new capabilities and by the time 
the capabilities are delivered, they may be out of date. Implementing knowledge- 
based evolutionary acquisition practices — where individual program increments are 
defined on the basis of mature technologies and a feasible design that incorporates 
an open architecture approach — would allow DOD to more effectively manage tech- 
nology changes in acquisition programs. Shorter, more manageable development cy- 
cles would also provide the warfighter with useful technologies quicker and provide 
more confidence that technologies can be developed within program cost estimates. 

Dr. Gansler. See my answer to question #13. 

Dr. Kaminski. I have always found that the mark of a good program manager is 
a large lower right hand desk drawer. In this drawer are placed all the ideas for 
incorporation of new technology advancements until the first version of the system 
is fielded. At that time the drawer is opened, and representatives from the require- 
ments and acquisition communities meet and apply systems engineering tools and 
techniques to assess shortfalls in the fielded system, evolving needs, and evolving 
technologies to examine value propositions and decide what to incorporate in the 
next block of that fielded system. 

Mr. Adolph. With platforms and major systems being procured less frequently 
and taking many years in development, there is often a tendency to over-reach since 
another opportunity to provide advanced technology to the warfighter may not occur 
for many years. The first, and most, beneficial action would be to work toward 
shortening the acquisition cycle. For various reasons, the acquisition cycle has be- 
come overly extended, due in part, as discussed in my response to question #13, to 
initiating acquisition programs based on immature technologies. This trend must be 
reversed. Recommendations have been made repeatedly in numerous studies that 
would result in a shortened cycle. 

One approach to a shortened cycle for large programs would be to incorporate new 
technologies in “block upgrades.” This approach has been successful for many de- 
fense systems, sometimes as a result of necessity and sometimes as preplanned im- 
provements. The latter approach has been implemented on many aircraft programs 
where planned multi-staged improvement programs are executed at regular inter- 
vals. Strategies such as spiral acquisition and pre-planned product improvement 
have been pursued with various de^ees of success. If too many new technologies 
are incorporated into a system, the risk is compounded by the complex task of inte- 
grating advanced technologies with sometimes insufficient technical maturity. These 
risks may be mitigated through disciplined processes to properly evaluate tech- 
nology maturity, larger investments in technical domain expertise as well as in sys- 
tems engineering. More emphasis should also be placed on analysis of alternatives, 
to include an OSD-level review. Designs that support “plug and play,” e.g. standard 
interfaces are also a partial solution. Producing the technology and integrating it 
into the logistics system as well as training maintenance personnel, et cetera, can 
still be challenges if the technology has never been deployed to the warfighter. (See 
also my response to question #13.) 


acquisition personnel 

21. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Ad- 
olph, a consistent theme throughout the hearing was the insufficiency of the current 
acquisition workforce. What would your recommendation be for increasing the num- 
ber of Active Duty billets specifically for acquisition? 
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Mr. Sullivan. Although we did not examine DOD’s decisionmaking process for 
using military versus civilian personnel, we recently reported that DOD lacks crit- 
ical Department-wide information to determine the sufficiency of its acquisition 
workforce to meet its mission needs. We recommended that DOD identify the num- 
ber and skill sets of its total acquisition workforce — including civilian, military, and 
contractor personnel — and conduct analyses using this information to inform acqui- 
sition workforce decisions regarding the appropriate mix of civilian, military, and 
contractor personnel. We are encouraged by the Department’s recent announcement 
that it plans to significantly increase the size of DOD’s acquisition workforce by con- 
verting 11,000 contractors and hiring an additional 9,000 government acquisition 
professionals by 2015 — beginning with 4,100 in fiscal year 2010. 

Dr. Ganslee. In the Commission I chaired (the “Commission on Army Acquisition 
and Program Management in Expeditionary Operations”) we stressed that it is not 
just numbers that matter, but rank as well as experience. The senior military acqui- 
sition positions should be staffed with General Officers that have extensive acquisi- 
tion experience. As to numbers, we should at least get back the 25 percent that Con- 
gress mandated be cut in fiscal year 1996 (which, across the DOD acquisition work- 
force, military and civilian, is approximately 50,000 additions — of which perhaps 20 
percent are military). 

Dr. Kaminski. Numbers alone aren’t the answer. We need an appropriate mix of 
quality and quantity. We need personnel able to provide early and continuing sys- 
tems engineering and closely-coupled development planning. We need: a) sufficient 
personnel (in both government and industry) with adequate education, training and 
domain experience (this includes personnel in requirements development as well as 
in acquisition); and b) sufficient front end investment in the tools necessary to un- 
derstand the key tradeoffs in cost/schedule/performance, and to identify and address 
the key risks in a systematic manner. 

Mr. Adolph. As a minimum, the government acquisition workforce, which in- 
cludes military and civilian personnel, should be increased to the levels that existed 
in the mid-1990s prior to the congressional and Service reductions. The military/ci- 
vilian acquisition position mix should be left to the individual Services as each is 
procuring different commodities, and has a different philosophy regarding a career 
military acquisition workforce. As I noted in my opening statement, during a time 
of increased programmatic and technical complexity, there has been a loss of a large 
number of the most experienced management and technical personnel, without an 
adequate replacement pipeline. Solutions to the acquisition problems must begin 
with reconstituting a trained and experienced government workforce, which includes 
programs managers, domain subject matter experts as well as systems engineers, 
contracts personnel, testers and evaluators. 

22. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Ad- 
olph, to ensure we get the quality and expertise we require, how do you recommend 
we recruit and retain this force? 

Mr. Sullivan. DOD has just announced plans to substantially increase its in- 
house acquisition workforce over the next few years. Our recent report on DOD’s 
acquisition workforce included practices of leading companies that could provide in- 
sights for DOD as it moves forward to hire and retain these employees. We found, 
for example, that the companies employed a variety of recruitment and retention 
initiatives. More importantly, some companies assessed their efforts at filling work- 
force gaps by tracking data on specific recruiting and retention metrics. One com- 
pany assessed the quality of their hiring sources by assessing the performance of 
new hires over their first 2 years. 

Dr. Gansler. Promotion potential and acquisition-experienced leaders are the 
keys to recruiting, developing, and retaining top acquisition personnel. 

Dr. Kaminski. You are quite right in noting that we must address retention as 
well as recruitment to build the requisite work force. We want people who want to 
make a difference. When we recruit these people, we must expect that we won’t 
keep them if they can’t see that they in fact are making a difference. Excessive (and 
growing) time from program initiation to fielding is a big problem for young people 
who want to see the impact of their work. As this time increases from a few years 
to 15 years or more, it undermines the entire acquisition process by causing key 
participants to “lose the recipe”, and lose a sense of accountability as well as a sense 
of being able to make a difference. When new capabilities are developed and fielded 
in 5 years, engineers, managers, testers, cost analysts, et cetera, are able to benefit 
from and apply the experience gained from a previous program or program phase. 
They can also see the results of their decisions and be held accountable. We can 
also meaningfully employ past performance of the contractor as a factor in the 
award of future programs — an important factor in incentivizing contractor perform- 
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ance. This all changes dramatically when the time extends to 15 years, and we have 
five roll-overs of management, engineers, cost analysts, and commercial technology 
during this time period. This long and growing time period is a result of the inflexi- 
bility inherent in our entire system of requirements development, budgeting, and ac- 
quisition, and it creates a vicious cycle in which it further exacerbates the contribu- 
tors above, and they in turn further increase the time and cost growth. We see the 
result when we must discard our current acquisition system in order to deal with 
urgent needs and field systems such as MRAP and jammers to counter lEDs by 
forming and using rapid reaction organizations. This cycle must be broken by at- 
tacking the root causes. 

Mr. Adolph. When the acquisition workforce reductions were made in the late 
1990s, they were implemented by encouraging early retirements as well as hiring 
freezes, which often lasted for years. As a consequence of the latter, the pipeline 
of new civilian employees was shut off, and today there are large gaps in experience 
as well as inadequate numbers of personnel. Some Service test organizations are ag- 
gressively addressing these issues and have more insight into what works in today’s 
environment than I do. Based on my past experience, initial hiring, particularly for 
positions at remote test locations, is the biggest challenge. It requires aggressive re- 
cruiting at universities, along with co-op programs. The challenge of the work envi- 
ronment and the learning experiences, dealing with the latest technologies, com- 
bined with well-funded full-time advanced degree educational opportunities (with 
associated follow-on career commitments), are powerful positive retention forces 
after people are brought on board. On-the-job learning experiences have been dimin- 
ished in some test organizations, where government personnel have been relegated 
to less challenging supporting functions. 

23. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Ad- 
olph, what can we do in the meantime to handle the personnel shortage since it will 
take many years to develop the level of expertise required for proper acquisition re- 
form? 

Mr. Sullivan. As we recently reported in our review of DOD’s acquisition work- 
force management, DOD is currently implementing initiatives aimed at improving 
the capacity of its acquisition workforce using the newly established Acquisition 
Workforce Development Fund. These efforts are focused in three key areas: (1) re- 
cruiting and retention; (2) training and development; and (3) retention and recogni- 
tion. DOD has also recently announced plans for significantly increasing its in-house 
acquisition workforce over the next few years. While DOD integrates these addi- 
tional employees into its acquisition workforce, the Department can also leverage 
the experience and knowledge of its current workforce. DOD can take steps to retain 
experienced personnel to aide in the development of newer staff and to ensure it 
maintains and transfers key institutional knowledge. 

Dr. Gansler. To fill the gap, while developing a new group, I would bring into 
the government some middle and senior people from industry; and I would use con- 
tractors (with experience, but without conflicts) to fill the rest of the slots. 

Dr. Kaminski. We can begin to hire people with education in engineering and sys- 
tems engineering. We can send our current personnel back to universities to en- 
hance their education. We can send our current personnel to industry to gain do- 
main experience. We can bring back retirees on a part time basis to provide men- 
toring to our new hires and help them gain domain experience. We need to do all 
of these. 

Mr. Adolph. During the period when the normal pipeline is being replenished, 
Centers of Excellence must be created in selected test and acquisition engineering 
support organizations, with a focus on technical domain subject matter expertise. 
These cadres of expertise will provide the basic oversight functions such as those 
I outlined in my answer to question #16. The first step is to provide oversight in 
key technical disciplines at key programmatic times, e.g., REP preparation, source 
selection, TEMP preparation, test reporting to support program reviews; 
transitioning to more active, continuous involvement as the workforce increases. 
The Services may be reluctant to take this approach. These Service cadres can also 
be augmented by subject matter experts from FFRDCs. The use of red teaming by 
outside experts (e.g., cross-Service, FFRDCs, and SETAs) should also be employed 
to provide the requisite expertise, as well as a measure of independence which is 
too often lacking in government program office-centric reviews. 
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MANDATORY PROCEDURES FOE MAJOR DEFENSE ACQUISITION PROGRAMS 

24. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Ad- 
olph, what oversight and reporting mechanisms do you recommend we establish to 
ensure the Mandatory Procedures for Major Defense Acquisition Programs (MDAPs) 
meet warfighter requirements while eliminating redundancy and wasteful spending? 

Mr. Sullivan. DOD has recently revised its acquisition policy in ways intended 
to provide key Department leaders with the knowledge needed to make informed de- 
cisions before a program starts and to maintain disciplined development once it be- 
gins. The revised policy recommends the completion of key systems engineering ac- 
tivities before the start of development, requires early prototyping, establishes re- 
view boards to identify and mitigate technical risks, and implements early milestone 
reviews. If implemented, these policy changes could help programs replace risk with 
knowledge, thereby increasing the chances of developing weapon systems within 
cost and schedule targets while meeting user needs. To achieve improved outcomes 
in acquisition programs, DOD must ensure its policy changes are consistently imple- 
mented and reflected in decisions on individual programs. However, DOD could do 
more by requiring new programs to have manageable development cycles and estab- 
lish knowledge-based cost and schedule estimates. Limiting the length of develop- 
ment cycles would make it easier to more accurately estimate costs, predict future 
funding needs, effectively allocate resources, and hold decision makers accountable. 

The acquisition reform legislation recently proposed by the Senate Armed Services 
Committee should also help achieve further improvements. Provisions increasing 
the emphasis on systems engineering, requiring early preliminary design reviews, 
and strengthening independent cost estimates and technology readiness assess- 
ments should make the critical front end of the acquisition process more disciplined. 
Establishing a termination criterion for critical cost breaches could help prevent the 
acceptance of unrealistic cost estimates at program initiation. In addition, having 
greater COCOM involvement in determining requirements and greater consultation 
between the requirements, budget, and acquisition processes could help improve 
DOD’s efforts to balance its portfolio of weapon system programs. 

While policy revisions and legislation may lead to improvements, they will not be 
effective without changes to DOD’s overall acquisition environment and the incen- 
tives that drive the behavior of its decisionmakers, the military Services, program 
managers, and the defense industry. Changing the environment will require strong 
leadership and accountability within the Department. 

Dr. Gansler. I believe we currently have enough oversight and reporting mecha- 
nisms for MDAPs (and more would only slow the system down even more). The crit- 
ical need is to use these mechanisms more effectively — with experienced personnel 
(who know what to look for, and what questions to ask). 

Dr. Kaminski. We first need to realize that “one size does not fit all.” For exam- 
ple, programs dealing with countering lEDs need cycle times measured in weeks, 
while a next generation bomber program must plan for fielding and support cycles 
measured in years. 

In most cases today the program manager’s authority is diffused by many levels 
of oversight in both the Department and in Congress, and the financial and per- 
formance constraints imposed do not allow sufficient freedom of action to apply in- 
formed judgment in a timely manner. Flexibility is further limited by application 
of a “one-size-fits-all” approach imposed by the DOD 5000 system, and the oversight 
practiced by the DOD and Congress. A program manager needs the freedom to tai- 
lor the acquisition approach to the problem, to ensure that the program response 
time will fall within the response time of the threat, and to apply a variety of tools 
and techniques (such as the use of prototypes, competitive prototypes, M&S, critical 
subsystem and component demonstration). For this to work, we need program man- 
agers with the education, training, and domain experience needed to enable timely 
responses and excellent judgment relevant to the domain. They need the flexibility 
to apply their expertise. 

Mr. Adolph. I believe the current oversight and reporting mechanisms, when 
combined with the changes in the proposed legislation, will provide adequate direc- 
tion and guidance for the acquisition process. The key is putting more discipline in 
every step of the process by the Services and OSD. This was achieved in numerous 
past successful programs in the 1960s, 1970s, and 1980s without being overly bu- 
reaucratic. One impediment is the recent trend towards contractual vehicles which 
relieve development contractors of reporting requirements, including regular status 
reports on costs and technical performance. The government must have continued 
access to cost and performance data to provide effective oversight. 

Most of the basic policies and directives for the acquisition process represent best 
practices based on decades of experience. The fundamental problem is a lack of ade- 
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quate discipline throughout the process. This begins with proper definition of oper- 
ational requirements at the outset of a program and stating those requirements in 
terms that are reasonable and quantifiable for design and test purposes. Another 
major issue is the lack of adequate technology maturity (See my response to ques- 
tion #13.) 

There have been some problems with military specifications and standards in the 
past, in that they were not properly tailored to the system to be procured. However, 
these standards have evolved and were updated over decades and were, for the most 
part, excellent compendiums of best practices and lessons learned. If properly used, 
they provide guidance for system development to ensure a systematic and dis- 
ciplined approach. Proper application of these guidance documents also ensures that 
problems, which are a normal part of any complex high technology development, are 
identified early. Many of these documents have been allowed to atrophy over the 
last several years or have been eliminated. Previously eliminated specifications and 
standards should be selectively updated and reinstated; while retaining the option 
to use commercial specifications and standards when available, provided the latter 
adequately address the requisite military performance and suitahility requirements 
in the intended operating environment. Finally, the most important ingredient re- 
mains a capable and experienced government acquisition workforce with equally ca- 
pable leaders, who remain on the job long enough to realize the consequences of 
their programmatic decisions. 


Questions Submitted by Senator David Vitter 

PERSONNEL IN ACQUISITION REFORM 

25. Senator Vitter. Dr. Gansler, a recurring message from the panel is the impor- 
tance of improving the quality and experience of personnel in acquisition reform, 
and I’m glad that Chairman Levin and Ranking Member McCain included section 
206 in their acquisition reform bill to reward excellence. This section requires the 
establishment of a program recognizing excellent performance and the award of 
cash bonuses. However, it was stated that cash isn’t necessarily the greatest incen- 
tive for encouraging excellence, and making a difference is the best incentive. In 
your opinion, what can DOD implement in an excellence program to demonstrate 
improvements so that personnel can track and actually see their work reflected in 
tangible results to provide and maintain motivation? 

Dr. Gansler. Combining appropriate metrics with rewards for success (including 
recognition, and even token financial rewards) are desirable — but the managers 
must be allowed to have sufficient flexibility to exercise their judgment, if they are 
to succeed. 

26. Senator Vitter. Dr. Gansler, how much of the problem in attracting the right 
quantity of qualified acquisition personnel is a product of the overall Federal em- 
ployee hiring rules established by Office of Personnel Management (0PM) or the 
specific rules established by DOD? 

Dr. Gansler. The DOD must be able to compete with industry for “the best and 
brightest.” The process must be flexible and fast — yet fair. Salary will not be the 
driver (government jobs offer the rewards) but the process can be neither bureau- 
cratic nor political. 

27. Senator Vitter. Dr. Gansler, last year. Congress authorized $300 million to 
rebuild the acquisition workforce. Do you believe this is sufficient? 

Dr. Gansler. It is an important start, and will likely need to be continued for 
a few years. 

28. Senator Vitter. Dr. Gansler, cost overruns were endemic in the 1980s and 
1990s when the acquisition workforce was robust. Is it likely then that adding more 
people will solve the problem of cost overruns? 

Dr. Gansler. As I noted in question #21, numbers alone are not the answer, sen- 
ior and experienced people with an improved acquisition process (as covered in my 
answers above — especially numbers 1, 8, and 13). 

29. Senator Vitter. Dr. Gansler, it seems that there is a problem with keeping 
quality program managers in their roles for an extended period of time. How can 
we ensure continuity at these positions? 

Dr. Gansler. Goldwater-Nichols intended the program managers to remain at 
least through the next milestone. In the Packard Commission we emphasized that 
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they be allowed to be promoted in place (so that the extended period is a reward, 
not a career-limiting assignment). 


FUNDING FOE DEVELOPMENTAL TESTING 

30. Senator Vitter. Mr. Adolph, to what extent does a lack of funding for develop- 
mental testing (DT) early in a program’s schedule result in cost increases and delays 
later in its schedule? 

Mr. Adolph. The lack of adequate funding for DT has a major adverse impact 
on developmental cost and schedule. There are numerous issues relating to DT 
funding. The weapons timelines for RDT&E continue to increase, driven by inad- 
equate resourcing of the entire developmental process as well as the inclusion of im- 
mature technologies in systems. Inadequate funding of DT, which includes the num- 
ber of test assets, results in delayed identification and correction of problems, many 
of which are a normal part of a high-technology developmental program. Late identi- 
fication results in more difficulty and expense involved in fixes. Most recent pro- 
grams have had an inadequate number of assets to execute a robust DT program 
in a timely manner. When there are an inadequate number of test assets, the DT 
program is drastically impacted, often for months, when a test vehicle must be laid 
up for a retrofit. By the time a program reaches the full-scale platform test phase, 
there is a huge cadre of test and test support personnel who cannot be efficiently 
used whenever a test program comes to a halt. The fixed cost of maintaining this 
cadre often exceeds the incremental variable costs of conducting test missions. An- 
other factor in efficient testing is the availability of adequate test personnel and test 
facilities. The latter includes physical test support facilities and assets, instrumenta- 
tion, and data processing (see also my response to question #17). Numerous develop- 
mental programs have been delayed because of the lack of adequate facilities, capac- 
ity, and people to accomplish the basic data processing tasks quickly, as well as in- 
adequate number of domain subject matter experts to analyze test results. This is 
exacerbated when earlier involvement by DT&E and OT&E personnel is limited by 
a lack of sufficient experienced personnel. Commercial programs routinely resource 
facilities and personnel to conduct testing on a multiple shift basis during critical 
phases of the developmental process. 


COMPETITIVE prototyping 

31. Senator Vitter. Dr. Kaminski, how much can competitive prototyping help in 
identifying the best product for warfighters? 

Dr. Kaminski. Competitive prototyping is one arrow in our quiver of tools and 
techniques in development planning. Not all prototyping needs to be competitive. 
Prototyping should not be limited to the full system level. There are cases where 
prototyping is best applied only to address major platform integration risk areas, 
or to critical subsystems. Again, I don’t believe there is a “one-size-fits-all” solution 
to prototyping. But I do believe that at a big picture level we are not doing enough 
prototyping in our development planning. Reducing risk and developing domain ex- 
pertise require that we build things. M&S are of great value, and so are our com- 
puter based analysis and design tools, but all of my experience tells me that we 
need to build and test on a continuing basis to make mistakes, learn from those 
mistakes, and apply that learning in the development of new capabilities. 

32. Senator Vitter. Dr. Kaminski, the F-16 program, lauded by many as an ex- 
ample of how to acquire a capability, was competed to the point where the Air Force 
actually flew competing planes before deciding on which defense contractor won the 
award. Why have we gotten so far away from that? 

Dr. Kaminski. We haven’t completely departed from that approach. We did some- 
thing very similar in the JSF program. But I agree that we are not doing enough. 
We can do quite a bit more. I believe we can do so without increasing costs by 
spending money upfront to reduce or avoid the cost of significant overruns later. 


JOINT requirements OVERSIGHT COUNCIL 

33. Senator Vitter. Mr. Sullivan, the Vice Chiefs have important jobs internally 
managing their respective Services. Do you believe that JROC should instead be 
staffed with independent personnel who are not advocates for a particular Service? 

Mr. Sullivan. GAO has found DOD’s requirements process to be too Service-cen- 
tric. The needs of the COCOMs should be given greater consideration. There are a 
number of ways the requirements setting process might be changed to address this 
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issue. For example, the combatant commanders might supplant the Service Vice 
Chiefs role in the process or might be given an equal say in the process. Several 
outside reviews have suggested such solutions: 

• The Beyond Goldwater-Nichols study team recommended a more “joint” 
JROC in which the Service Vices are replaced by COCOM Deputies and ci- 
vilians responsible for requirements policy. 

• The Defense Acquisition Performance Panel (DAP A) recommended replac- 
ing the current requirements setting process with a combatant commander- 
led requirements process in which the Services and defense agencies com- 
pete to provide solutions. Each of the combatant commanders would be 
tasked to prepare extended planning annexes to each of their operational 
and contingency plans (to be updated on a 2-year cycle) that would provide 
a 15-year forecast of both capability gaps and excesses relative to mission 
requirements. 

• Some members of the Defense Science Board Summer Study on Trans- 
formation felt that the requirements process continues to be dominated by 
the force providers and the Joint Staff and that COCOM needs are under- 
represented. They proposed making the COCOMs more equal partners with 
the force providers from the beginning of the process, particularly when 
identifying capabilities needed to carry out the Department’s operational 
missions. 

We believe that these proposals deserve further consideration as a means to help 
improve the Department’s ability to balance joint warfighting needs. 

34. Senator Vittee. Mr. Sullivan, should we have a new, independent JROC that 
could hear all advocates for all programs and then recommend continuation or elimi- 
nation of the program based on current and likely future threats to the United 
States? 

Mr. Sullivan. Because many major defense acquisition programs have ambitious 
and lengthy product developments — in some cases, delivery of a weapon system to 
the warfighter takes as long as 15 years — there can be a significant amount of time 
that elapses between JROC reviews and changes in the threat environment can 
occur. Before a major weapon program is approved to begin system development, the 
JROC must validate the capabilities to be developed, including the system’s key per- 
formance parameters. The JROC also validates the production elements of a pro- 
gram before it is approved to start production. For programs that have lengthy de- 
velopment cycles, more frequent reviews may be necessary to assess whether pro- 
grams are worth continuing in light of current and likely future threats. In prior 
work we have conducted on how successful companies manage their portfolios of 
product development efforts, we found that the companies revisit their investment 
decisions at multiple stages throughout product development to ensure products are 
still of high value. 

35. Senator Vittee. Mr. Sullivan, should the Services be required to budget their 
programs to the cost estimates provided by a new Director of Independent Cost As- 
sessment? 

Mr. Sullivan. Budgeting programs to an independent cost estimate would be an 
improvement, however, for cost estimates to be effective they must be based on a 
high degree of knowledge about requirements, technology, design, and manufac- 
turing. 

A reliable cost estimate helps ensure a program’s projected funding needs are ade- 
quate to execute the program. Less than a quarter of the 48 programs in CAO’s an- 
nual assessment of weapon system programs that provided data used the estimate 
made by DOD’s Cost Analysis Improvement Croup (CAIC) as a basis for the pro- 
gram’s baseline, while almost 70 percent of the programs used the program office 
or Service cost estimate. While cost estimates from the CAIC can underestimate a 
program’s costs by billions of dollars, CAO has previously found that these inde- 
pendent estimates generally underestimate costs by a smaller amount than program 
office and Service estimates. 

Cost estimates are inaccurate in part because they are based on limited knowl- 
edge about the requirements, technologies, design maturity, and the time and fund- 
ing needed to execute a program. CAO has found, for example, that program Cost 
Analysis Requirements Documents — used to build the program cost estimate — often 
lack sufficient detail about planned program content for developing sound cost esti- 
mates. Without this knowledge, cost estimators must rely heavily on parametric 
analysis and assumptions about system requirements, technology, and design matu- 
rity, and the time and funding needed. The assumptions used in developing esti- 
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mates also tend to be overly optimistic. Furthermore, cost estimates that lack 
knowledge and rely on assumptions have inherently high levels of risk and uncer- 
tainty that are not typically communicated to decision makers. 

36. Senator Vitter. Mr. Sullivan, with the annual Selected Acquisition Reports 
and current Nunn-McCurdy reporting requirements, is Congress suffering from a 
lack of oversight and reports from DOD? 

Mr. Sullivan. In terms of the reports it receives from DOD, Congress is suffering 
from a lack of insight into the risks on programs that could lead to cost, schedule, 
and performance shortfalls. The Selected Acquisition Reports and Nunn-McCurdy 
reporting that Congress gets from DOD on its major defense acquisition programs 
primarily report on program outcomes to date — including quantitative measures of 
cost, schedule, and performance over time. While these are useful indicators of the 
health of acquisition programs and whether they are meeting their intended goals, 
they have limited predictive value. DOD could improve the information it provides 
to Congress by reporting on knowledge metrics that evaluate whether programs 
have attained certain levels of technology, design, and manufacturing knowledge by 
key points in the acquisition process. This reporting would facilitate the identifica- 
tion of potential problems that could lead to cost, schedule, and performance short- 
falls before they occur. 

37. Senator Vitter. Mr. Sullivan, is there any credence to the concern that pro- 
gram managers have “too many” reports to file and are not focusing on managing 
their programs? 

Mr. Sullivan. Oversight and reporting requirements need to be evaluated both 
from the perspective of the time they take and the value they provide. In the past, 
we have reported on concerns in both of those areas. In a November 2005 report, 
we reported that program managers and program executive officers commented that 
they spend too much time producing data that is not strategic or very useful to 
them. In fact, more than 90 percent of 126 respondents to our survey of program 
managers said that they spent either a moderate, great, or very great extent of their 
time representing their program to outsiders and developing and generating infor- 
mation about program progress. 

We have also found that the reporting that is done does not include key informa- 
tion that decisionmakers need to decide if a program is ready to proceed into the 
next phase of the acquisition process. For example, in transitioning from system in- 
tegration to system demonstration, we have recommended that DOD ensure the cap- 
ture of knowledge about the completion of engineering drawings; completion of sub- 
system and system design reviews; agreement from all stakeholders that the draw- 
ings are complete; and identification of critical manufacturing processes, among 
other indicators. In the transition to production, we recommended that DOD capture 
knowledge about production and availability of representative prototypes along with 
statistical process control data. Our 2006 report indicated that a relatively small 
percentage of programs used these knowledge indicators. For example, only 32 per- 
cent of 126 program managers who responded to our survey said they used design 
drawing completion extensively to measure design maturity; only 26 percent said 
they used production process controls to a great extent. Even fewer program man- 
agers reported that their immediate supervisor used these measures extensively to 
evaluate progress. Broader use and reporting of these types of metrics would help 
decisionmakers gauge progress and ensure that programs managers are: (1) con- 
ducting activities to capture relevant product development knowledge; (2) providing 
evidence that this knowledge has heen captured; and (3) holding decision reviews 
to determine that the requisite knowledge has been captured before proceeding to 
the next phase of the acquisition process. 

[Whereupon, at 11:52 a.m., the committee adjourned.] 
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